# AISFOR-CHAIN Ledger DB Migration Plan

## Current State
- Current ledger backend: JSON file (`data/ledger.json`).
- DB backend switching is intentionally disabled.
- Admin visibility is available at `/admin/ledger-migration`.

## Migration Order
1. Confirm DB engine and production URI.
2. Build a dry-run backfill script from JSON arrays to DB collections.
3. Compare row counts and hashes after dry-run.
4. Enable dual-write so JSON and DB receive the same new ledger events.
5. Run dual-write observation until counts remain stable.
6. Switch reads to DB behind an environment flag.
7. Keep JSON as rollback snapshot only.

## Collection Map
- proofs -> chain_proofs
- transactions -> chain_transactions
- blocks -> chain_blocks
- callbackEvents -> chain_callbacks
- nodes -> node_participants
- nodeEconomyEntries -> node_economy_ledger
- pointEntries -> point_ledger
- validatorVotes -> validator_votes
- backupIndexes -> backup_indexes
- distributedChunks -> distributed_chunks
- wallets -> wallets
- walletTransactions -> wallet_transactions

## Safety Rules
- Do not delete `ledger.json` during migration.
- Do not enable confirmed DB switching until dry-run and dual-write are verified.
- Do not store raw private data in chain ledger collections.

## 2026-06-30 Dry-run Preparation
- Added a DB migration dry-run generator.
- Dry-run output is written to `data/migration-dry-run.json`.
- Admin execution is available at `/admin/ledger-migration` with the `Dry-run 생성` button.
- CLI execution is available with `node scripts/ledger-db-dry-run.js`.
- Dry-run creates deterministic `_migrationId` and `_sourceHash` values per row without writing to a DB.
- DB writes and backend switching remain locked until a real DB URI and dual-write observation period are approved.

## 2026-06-30 Shadow Dual-write Preparation
- Added `services/dbLedgerAdapter.js` as a DB-shaped shadow adapter.
- Default production behavior remains unchanged because the adapter is inactive unless both flags are enabled.
- Required flags: `AISFOR_CHAIN_LEDGER_DUAL_WRITE=1` and `AISFOR_CHAIN_LEDGER_DB_WRITE=1`.
- Shadow output is written as per-collection JSON files under `data/db-shadow`.
- `persistentStore.writeState()` mirrors safely with failure isolation, so a shadow write failure cannot block `ledger.json`.
- Admin visibility was added to `/admin/ledger-migration`: shadow adapter, shadow docs, shadow sync, and shadow dir.
- CLI validation is available with `node scripts/ledger-dual-write-shadow-test.js`.
- This is not the final DB backend. It only proves collection mapping and dual-write shape before real DB adoption.

## 2026-06-30 Shadow Consistency Check
- Added `services/ledgerConsistencyService.js`.
- Added CLI comparison with `node scripts/ledger-shadow-compare.js`.
- The checker compares source `ledger.json` rows against `data/db-shadow` rows by collection count and source hash sequence.
- Admin visibility was added to `/admin/ledger-migration` under `Shadow Consistency`.
- This is the gate before attaching a real DB writer.

## 2026-06-30 DB Backend Readiness Gate
- Added `services/ledgerDbReadinessService.js`.
- Added CLI inspection with `node scripts/ledger-db-readiness.js`.
- Admin visibility was added to `/admin/ledger-migration` under `DB Backend Readiness`.
- The gate checks rollback snapshot, dry-run output, shadow consistency, DB URI, DB driver, and read switch lock status.
- DB read switching remains disabled even when the writer prerequisites pass. A separate dual-write observation approval is still required.
- Current expected development state is that MongoDB writer attachment stays locked until a DB URI and `mongodb` driver are intentionally added.

## 2026-06-30 MongoDB Staging Backfill
- Added MongoDB driver dependency and configured the development ledger DB URI.
- Added `scripts/ledger-db-probe.js` to verify DB connectivity without writing data.
- Added `scripts/ledger-db-stage-backfill.js` for staging-only MongoDB backfill.
- Added `scripts/ledger-db-stage-compare.js` for count and source-hash comparison.
- Staging collections use the `migration_staging_*` prefix and do not touch live ledger collections.
- Latest staging run wrote 212 documents and the compare step matched 212 / 212 with 0 failed collections.
- Staging summaries are written to `data/migration-staging-latest.json` and `data/migration-staging-compare.json`.
- Admin visibility was added to `/admin/ledger-migration` under `MongoDB Staging`.
- Writer attachment prerequisites are now satisfied, but DB reads remain locked until a separate dual-write observation approval.

## 2026-06-30 MongoDB Observation Dual-write
- Added `services/mongoLedgerObservationAdapter.js`.
- Added `scripts/ledger-mongo-observation-sync.js` and `scripts/ledger-mongo-observation-compare.js`.
- Observation writes use `observation_*` collections and do not touch live ledger collections.
- Enabled `AISFOR_CHAIN_LEDGER_MONGO_OBSERVE=1` and `AISFOR_CHAIN_LEDGER_DB_WRITE=1`.
- `persistentStore.writeState()` now mirrors JSON ledger writes to MongoDB observation collections asynchronously after JSON write success.
- A no-op ledger write refreshed MongoDB observation sync successfully.
- Latest observation compare matched 212 / 212 documents with 0 failed collections.
- Admin visibility was added to `/admin/ledger-migration` under `MongoDB Observation`.
- DB reads remain locked. The next gate is an observation window across real service writes.

## 2026-06-30 Observation Window Runner
- Added `scripts/ledger-mongo-observation-window.js`.
- Admin can trigger a fresh observation compare with `/admin/ledger-migration/observation-compare`.
- `/admin/ledger-migration` now shows observation window samples, consecutive OK count, and ready/observing status.
- Latest short runner check reached 6 consecutive OK samples with 212 / 212 documents and 0 failed collections.
- This proves the observation mechanism works, but the read switch still requires a longer real-service activity window.

## 2026-06-30 Read Switch Preflight Gate
- Added `scripts/ledger-read-switch-preflight.js`.
- DB readiness now reports `readSwitchPreflightOk`, `readSwitchApproved`, and `canSwitchReads` separately.
- Current technical preflight is ready: DB URI, driver, dry-run, shadow consistency, Mongo observation compare, and observation window are all OK.
- `canSwitchReads` remains false because manual read-switch approval is still required.
- `/admin/ledger-migration` now shows read preflight, manual approval, read switch lock, and lock reason.

## 2026-06-30 Manual Read Switch Approval Control
- Added `services/ledgerReadSwitchApprovalService.js`.
- Manual read-switch approval is now persisted in `data/read-switch-approval.json` instead of relying only on environment variables.
- Added admin POST controls:
  - `/admin/ledger-migration/read-switch/approve`
  - `/admin/ledger-migration/read-switch/revoke`
- `/admin/ledger-migration` now shows approval id, approval time, approval file path, and approval/revoke buttons.
- Verification confirmed that a recorded approval changes `canSwitchReads` to true while no read backend switch is requested.
- The temporary verification approval was revoked, so live reads remain on JSON.

## 2026-06-30 MongoDB Read Backend Adapter
- Added `services/mongoLedgerReadAdapter.js`.
- Added `scripts/ledger-mongo-read-status.js` to report the Mongo read gate.
- Added `scripts/ledger-mongo-read-compare.js` to compare JSON ledger rows against live Mongo read collections.
- Added `scripts/ledger-mongo-live-backfill.js` for live collection backfill, locked unless `--confirm-live-backfill` or `AISFOR_CHAIN_LEDGER_LIVE_BACKFILL_APPROVED=1` is provided.
- The read adapter only reads documents marked with `_ledgerLiveKind = ledger-json-live-backfill`.
- Current verification result: JSON has 212 rows, live Mongo read collections have 0 rows, so live backfill is the next required gate before any read switch.
- Current live read backend remains JSON.

## 2026-06-30 MongoDB Live Backfill Verification
- Executed confirmed live Mongo backfill with `node scripts/ledger-mongo-live-backfill.js --confirm-live-backfill`.
- Wrote 212 documents into live Mongo read candidate collections.
- The live read adapter reads only rows marked with `_ledgerLiveKind = ledger-json-live-backfill`.
- Re-ran `node scripts/ledger-mongo-read-compare.js` after backfill.
- Compare result matched 212 / 212 with 0 failed collections.
- Read backend remains JSON because Mongo read request and manual approval are still not active.
- Next gate: run a controlled read-switch request test with approval, then revoke or keep locked depending on result.

## 2026-06-30 Controlled Mongo Read Gate Test
- Temporarily recorded manual read approval for a process-local test only.
- Ran a separate Node process with `AISFOR_CHAIN_LEDGER_READ_BACKEND=mongodb`.
- `mongoLedgerReadAdapter.readStateFromMongo()` passed the gate and read 212 documents from MongoDB.
- Revoked the temporary approval immediately after the test.
- PM2/service environment was not switched; live reads remain JSON.

## 2026-06-30 Mongo Read Snapshot Path
- Added `services/mongoReadSnapshotService.js`.
- Added `scripts/ledger-mongo-read-snapshot.js` to materialize a verified Mongo read snapshot after 212 / 212 compare succeeds.
- `persistentStore.readState()` can now use the Mongo-derived snapshot only when the Mongo read request and manual approval are both active.
- Process-local test confirmed `persistentStore.readState()` reads 212 rows from the Mongo snapshot under the gated read path.
- Temporary approval was revoked after the test; live service reads remain JSON.

## 2026-06-30 PM2 Mongo Read Switch Rehearsal
- Refreshed Mongo read snapshot after a successful 212 / 212 compare.
- Temporarily set `AISFOR_CHAIN_LEDGER_READ_BACKEND=mongodb` and recorded manual approval.
- Restarted PM2 `aisfor-chain` with the temporary Mongo read setting.
- Verified `/admin/ledger-migration`, `/admin`, `/explorer`, `/explorer/tx/:txHash`, and `/explorer/address/:address` returned 200 under `mongodb-read-enabled`.
- Rolled the environment back to `AISFOR_CHAIN_LEDGER_READ_BACKEND=json-file` and revoked the temporary approval.
- Restarted PM2 again and confirmed `json-read-active` with the Mongo snapshot still available as a verified fallback candidate.

## 2026-06-30 Read Backend Request/Rollback Controls
- Added `services/ledgerReadBackendSwitchService.js`.
- The service edits only `AISFOR_CHAIN_LEDGER_READ_BACKEND` in `.env`, creates timestamped backups, and records switch history in `data/read-backend-switch-history.json`.
- Added admin controls on `/admin/ledger-migration`:
  - `Mongo Read 요청`: records manual approval and sets the read backend request to `mongodb`.
  - `JSON Read 롤백`: sets the read backend request back to `json-file` and revokes approval.
- Verified the service can request Mongo mode and roll back to JSON mode without restarting PM2 automatically.
- Current `.env` read backend is `json-file`.

## 2026-07-01 Mongo Live Writer And Snapshot Refresh
- Added `services/mongoLedgerLiveWriterAdapter.js`.
- Added `scripts/ledger-mongo-live-sync.js` with explicit `--confirm-live-sync` gate.
- `persistentStore.writeState()` now refreshes the Mongo read snapshot synchronously when Mongo read is enabled.
- `persistentStore.writeState()` also triggers the Mongo live writer asynchronously when live write is explicitly enabled or Mongo read is active.
- Manual live sync was verified and wrote 212 documents.
- Post-sync compare matched 212 / 212 with 0 failed collections, and the read snapshot was refreshed to 212 documents.
- Current live read remains `json-read-active`; live writer is standby unless explicitly enabled or Mongo read is active.

## 2026-07-01 Mongo Read Monitor Gate
- Added `services/ledgerMongoReadMonitorService.js`.
- Added `scripts/ledger-mongo-read-monitor.js` for repeated Mongo-vs-JSON read checks.
- Added `/admin/ledger-migration/read-monitor/sample` so the admin migration page can collect a fresh read monitor sample without changing the active read backend.
- `/admin/ledger-migration` now shows latest sample status, expected/actual counts, failed collections, sample window, and consecutive OK count.
- The monitor uses the live Mongo read collections with `ignoreGate: true`, but it does not switch runtime reads away from JSON.
- Promotion rule: require at least 6 consecutive OK samples before treating Mongo read as operationally ready.

## 2026-07-01 Mongo Post Switch Guard
- Added `services/ledgerMongoPostSwitchGuardService.js`.
- Added `scripts/ledger-mongo-post-switch-guard.js`.
- Added `/admin/ledger-migration/post-switch-guard/run` and a Post Switch Guard panel.
- The guard checks JSON-vs-Mongo read parity and verifies `/admin`, `/explorer`, and `/admin/ledger-migration` HTTP health.
- By default the guard only records health. It prepares JSON rollback only when CLI is run with both `--rollback-on-failure` and `--confirm-rollback`.
- This provides the final safety check for a controlled Mongo read promotion without changing the live read backend by accident.

## 2026-07-01 Controlled Mongo Read Promotion
- Final preflight passed before promotion: DB URI, driver, dry-run, shadow consistency, Mongo observation, observation window, and rollback snapshot were all OK.
- Read monitor passed with 12 consecutive OK samples and 212 / 212 document parity.
- Recorded manual read-switch approval with actor `codex-final-promotion`.
- Set `AISFOR_CHAIN_LEDGER_READ_BACKEND=mongodb` through `ledgerReadBackendSwitchService`, preserving an `.env` backup and switch history entry.
- Restarted PM2 `aisfor-chain` with updated environment.
- Runtime read status is now `mongodb-read-enabled`.
- Post-switch guard passed: `/admin`, `/explorer`, and `/admin/ledger-migration` returned 200, Mongo-vs-JSON parity remained 212 / 212, and rollback was not prepared.
- JSON remains available as rollback source through the existing `JSON Read 롤백` control and explicit CLI rollback path.

## 2026-07-01 Completion Checkpoint
- AISFOR-CHAIN ledger read migration is complete at the operating level.
- Active runtime read backend is `mongodb-read-enabled`.
- Final monitor run passed 12 / 12 samples with 212 / 212 parity and 0 failed collections.
- Final post-switch guard passed with `/admin`, `/explorer`, and `/admin/ledger-migration` returning 200.
- Rollback remains available, but no rollback was prepared because all health checks passed.
- Remaining work is normal operational monitoring, not migration blocking work.
