v2 → v3 Offline Migration

wkcli migrate converts stopped original v2 data into native v3 data directories. You do not need to modify or upgrade the deployed v2: prepare reads backups offline on the migration machine; it is not an endpoint installed in v2.

API and SDK compatibility does not imply disk-format compatibility. Do not let v3 open a v2 data directory or replace this migration with a rolling upgrade.

Supported scope

The migration machine needs space for full cold backups, workspaces, the archive, every target replica, and snapshots taken before acceptance testing. Rehearse with representative data and record peak space and elapsed time. The 100 GiB / four-hour performance target has not been validated and is not a downtime guarantee.

1. Preserve cold backups and prepare the tool

If original v2 is running, stop every business write entrypoint, let log application, topology changes, and notification queues drain, then shut down all nodes normally and disable automatic restarts. Preserve complete node data directories, original binaries, configuration, and environment variables. Do not copy a database directory while it is being written.

An existing complete stopped backup can be used directly; do not restart v2 just to run prepare. Collect every source node. The tool checks file locks and compares file inventories and digests before and after scanning. A nonempty notification queue blocks completion; an empty queue does not prove that external systems received every old notification.

Build inside the fixed v3 source checkout. Use the Go version required by its go.mod:

umask 077
mkdir -p /srv/tools /srv/wkmigrate/reports /srv/wkmigrate/work /srv/wkmigrate/targets
GOWORK=off go build -o /srv/tools/wkcli ./cmd/wkcli
GOWORK=off go build -o /srv/tools/wukongim-v3 ./cmd/wukongim
git rev-parse HEAD > /srv/wkmigrate/reports/tool-source.txt
sha256sum /srv/tools/wkcli /srv/tools/wukongim-v3 > /srv/wkmigrate/reports/binaries.sha256
/srv/tools/wkcli migrate --help

On macOS, use shasum -a 256 instead of sha256sum. Build for the actual runtime platform. For a delivery package, compare its archive digest with an independent delivery record before checking the enclosed files. Since v3.0.0-beta.12, official archives and native packages include both binaries; see wkcli installation. Compare wkcli version --output json with wukongim version --output json: version, commit, and build source must match before migration.

2. Freeze the migration plan

Save this three-node example as /srv/wkmigrate/plan.json. Replace the nodes, paths, addresses, creation time, and actual source shard counts. source_commit is version evidence supplied by you; it does not automatically identify the old binary.

{
  "version": 1,
  "source_commit": "a888f89533d0e7d1b2030e06504ca97f1ad891d4",
  "sources": [
    {"node_id": 1001, "data_dir": "/srv/v2-snapshots/node1001/data", "shard_count": 8},
    {"node_id": 1002, "data_dir": "/srv/v2-snapshots/node1002/data", "shard_count": 8},
    {"node_id": 1003, "data_dir": "/srv/v2-snapshots/node1003/data", "shard_count": 8}
  ],
  "target": {
    "cluster_id": "migration-v3-new",
    "created_at": "2026-09-09T00:00:00Z",
    "slot_count": 256,
    "hash_slot_count": 256,
    "replicas": 3,
    "channel_replicas": 3,
    "nodes": [
      {"node_id": 101, "addr": "10.20.0.11:7000", "data_dir": "/srv/wkmigrate/targets/node101"},
      {"node_id": 102, "addr": "10.20.0.12:7000", "data_dir": "/srv/wkmigrate/targets/node102"},
      {"node_id": 103, "addr": "10.20.0.13:7000", "data_dir": "/srv/wkmigrate/targets/node103"}
    ]
  }
}

• Each sources[].data_dir points to the complete business database directory of one source node. Set shard_count to the actual business DB shard count in that deployment.
• For a single-node cluster, list one target node and set both replica counts to 1. Otherwise, replica counts cannot exceed the target node count. Target node IDs are 1–1023.
• Use absolute paths. Source directories, workspaces, the archive, and target directories must not contain one another. Create only their parents: new workspaces and target data_dir directories must not exist. Precreated empty directories are rejected because they lack migration identity.
• Keep the plan unchanged on retries. When changing sources, targets, policies, or tool versions, preserve the old attempt and start again with new workspaces, a new archive, and new targets.

Decide which data to import

The default checks business equivalence strictly. The following policies require explicit decisions; exceptions from one rehearsal must not be copied into another backup's plan without assessment.

If the business owner decides to omit CMD and entire stream messages, deduplicate, and renumber, merge these fields into the top level of the plan:

{
  "messages": {
    "keep_latest_duplicates": true,
    "exclude_cmd": true,
    "exclude_streams": true,
    "compact_sequences": true
  },
  "exclusions": {"legacy_stream_storage": true}
}

legacy_stream_storage only excludes the old Stream and StreamMeta tables. exclude_streams additionally excludes stream main messages from the message table and their explicitly associated event projections and cursors. Event identities shared with retained messages still block completion. exclude_cmd also omits old CMD conversations and sync positions. All excluded source rows remain in the archive.

“Latest” means the greatest original MessageSeq within the same channel. Keep the newer record for duplicate MessageIDs or the same channel with matching nonempty sender and nonempty ClientMsgNo. Empty senders or ClientMsgNo values skip ClientMsgNo deduplication; MessageID deduplication remains independent. Cross-channel MessageID conflicts and contradictory retention choices still block completion. Do not add physical replica counts across nodes to calculate business deletions.

After renumbering, retained messages become 1…N in their original order, and new messages continue at N+1. Read and deletion positions map to the number of retained messages at or before the old position. A channel with every message excluded has tail 0 and starts new messages at 1. The tool does not create placeholders or change v3 storage to fill gaps. Gaps already present in source history still block completion.

3. Run diagnose and prepare

Diagnose the cold backups in a separate workspace:

/srv/tools/wkcli migrate diagnose --plan /srv/wkmigrate/plan.json \
  --workspace /srv/wkmigrate/work/diagnose \
  > /srv/wkmigrate/reports/diagnose.json

Exit code 1 can indicate business blockers or an incomplete scan. Read the report and its full details first. Successful diagnosis does not select authoritative replicas or generate an importable target. Do not reuse the diagnosis workspace for preparation.

After resolving blockers and fixing the plan, run:

/srv/tools/wkcli migrate prepare --plan /srv/wkmigrate/plan.json \
  --workspace /srv/wkmigrate/work/prepare \
  > /srv/wkmigrate/reports/prepare.json

prepare checks original formats, indexes, and shards; uses durable configuration, Slot logs, and applied positions to determine sources; compares official replicas; and builds a conversion manifest. It does not choose the longest follower when replicas disagree. This step neither changes original v2 data nor starts or creates the target cluster.

Success requires exit code 0 and status: "prepared". Save the capture and selection digests, import/exclusion counts, and, when sequence conversion is enabled, the sequence_mapping file, row count, and SHA-256. Resolve authority, duplicate credential, missing conversation, and plugin blockers individually. See the engineering runbook for authority, dedupe-plan, and decisions bound to an exact capture. Do not delete source rows, clear fields, or rewrite proofs to pass checks.

4. Export the original archive

/srv/tools/wkcli migrate export --plan /srv/wkmigrate/plan.json \
  --workspace /srv/wkmigrate/work/prepare \
  --archive /srv/wkmigrate/source-archive \
  > /srv/wkmigrate/reports/export.json

Reuse the successful prepare workspace. Export rechecks that sources are unchanged and produces archive chunks, a manifest, and a COMPLETE marker. Missing chunks, checksum failures, or a missing completion marker prevent import. Once the archive and sequence map are preserved, source copies can be unmounted from the migration machine. Keep the complete original v2 cold backups.

5. Import and verify with targets stopped

Import and independent verification require only the plan and complete archive. Use separate new workspaces so verification does not depend on temporary import results:

/srv/tools/wkcli migrate import --plan /srv/wkmigrate/plan.json \
  --workspace /srv/wkmigrate/work/import \
  --archive /srv/wkmigrate/source-archive \
  > /srv/wkmigrate/reports/import.json

/srv/tools/wkcli migrate verify --plan /srv/wkmigrate/plan.json \
  --workspace /srv/wkmigrate/work/verify \
  --archive /srv/wkmigrate/source-archive \
  > /srv/wkmigrate/reports/verify.json

This version generates every target node directory on the migration machine. Import success means exit code 0 and status: "imported"; do not start targets yet. verify independently derives expected values from original archived records and compares every target replica field by field, including credentials, permissions, conversations, messages, native indexes, commit boundaries, and bootstrap artifacts. It checks more than counts.

Preserve the verification report and complete target snapshots before first startup. Once started, databases cannot be verified as initial import state or overwritten by another import.

Recovering an interrupted attempt

Keep the plan, archive, targets, and original logs. If targets have never started and the plan and archive are unchanged, rerun the same import command. Completed portions are fingerprint-checked; data from a different migration generation is not overwritten. Verification of unstarted targets can be rerun in full. Save each attempt's logs separately. Directories without migration identity are rejected; do not remove identity or completion markers to force reuse.

To rehearse the full offline flow in isolated Docker containers, use the repository's rehearsal scripts and examples. They provide read-only sources, separate workspaces, disk/time guards, and --dry-run. The wrapper does not automatically resume an existing output directory; inspect the attempt under the rules above before recovery.

6. Start v3 and run acceptance tests

Copy each complete target directory to its corresponding host, preserving the same absolute data path specified in the plan and file permissions. Include Controller snapshots and migration markers, not just message databases. Keep all targets stopped during copying.

Prepare each node's wukongim.toml. Match node.id, node.data_dir, cluster.id, cluster.nodes, cluster.initial_slot_count, cluster.hash_slot_count, cluster.slot_replica_n, and cluster.channel_replica_n to the plan. Separately check TLS, listening/advertised addresses, gateway authentication, Webhook endpoints and network protections, plugins, and business backend configuration. Database import does not transfer or validate those settings. Native Webhooks do not provide signatures; if your business uses an external signing proxy, migrate and validate that configuration separately. See Cluster Configuration and Security Configuration.

Start all targets in isolation using the matching v3 server, without production traffic. At minimum, check:

• /readyz succeeds on every node, and Controller, logical Slots, and channel replicas are healthy.
• Original tokens with the same device_flag authenticate; incorrect tokens are rejected. Do not reset tokens to “validate” old credentials.
• First, last, and cross-page history matches payloads, IDs, sequences, ClientMsgNo, and RedDot. Permissions, conversation lists, and read state match the plan.
• New message sequences strictly exceed the current channel tail, retrying the same idempotency key does not duplicate messages, and new unread counts are correct. For person sends, use the recipient UID, not the peer UID from history queried as the recipient.
• History, new messages, and unread state survive full-cluster restarts. For multi-node clusters, test single-node failure recovery under the agreed plan.
• Retained event projections, plugins, Webhooks, push notifications, and actual business clients pass their own acceptance tests.

Run write tests against isolated rehearsal copies. API checks do not replace SDK login, actual UI, and external integration acceptance. The Chat Demo supports testing with existing tokens; do not create or overwrite original credentials.

7. Client cutover and rollback boundaries

Renumbering invalidates old message caches and sequence cursors. For a cache-reset cutover, clear client history copies and derived sync cursors by migration generation, preserve login credentials, then sync conversations and history from v3. Handle unsent messages and drafts separately before clearing caches to avoid deleting or resending pending work. Server import does not clear the business App's database.

If clients use a mapping instead, they must handle the complete sequence_mapping. Excluded rows have target_seq 0; use boundary_seq for old cursors rather than treating excluded rows as existing messages. Rebuild the mapping from the archive with:

/srv/tools/wkcli migrate export-map --plan /srv/wkmigrate/plan.json \
  --workspace /srv/wkmigrate/work/map \
  --archive /srv/wkmigrate/source-archive

After all reports, runtime tests, and client acceptance pass, operators can switch routing and gradually restore traffic. Never allow both generations to accept writes for the same business simultaneously.

This version has no reverse incremental v3 → v2 migration. Rehearse rollback with backup copies without damaging original cold backups. See the delivery package acceptance report for a fixed-version migration and restart test. Its functional results do not certify your performance, plugins, or production cutover.