Data Migration Reconciliation and Cutover
Row counts matched on both sides at cutover. Month-end totals disagreed by eleven thousand. The count had been the only check anyone wrote down.
-
-
Parallel Run
In a fictional telecom, a parallel run compared month-end figures from both systems and exposed a rounding rule the legacy report had applied for nine years.
-
Control Loop
Continuous reconciliation can coexist with completed rollouts and Jobs. Observe progress, controller health, and service outcomes separately.
-
Desired State
A recorded target needs a working reconciler. Check completion, managed scope, and repeated effects rather than treating acceptance as success.
-
Enterprise Integration and Consistency
The message was delivered and the order was never shipped. Both statements can be true at once, and only one of them is what the business asked for.
-
Pipeline Correctness: Duplicates and Loss
Every job succeeded and the total is still wrong. Here is where duplicates and missing rows come from, what “exactly once” really promises, and how to verify.
-
Merkle Tree
A hierarchy of hashes lets two replicas locate exactly which ranges differ — without sending or comparing every row they hold.
-
Case: Data Platform Migration for AI Consumers
A fictional distributor replaces its warehouse while a search index and a prediction model depend on the old outputs. Contracts, dual running, reconciliation, and a coded cutover rule decide when each consumer moves.
-
Case: An Approved Agent for Transactional Work
A fictional company’s refund and reshipment agent, built and run locally: delegated credentials, approvals tied to one call, crash-safe steps, reconciliation with the ledger, and a policy-release incident drill.
-
Accepted Break
Not every difference found in a data migration is worth fixing. An accepted break records the ones the team will live with: who approved them, why, how far they extend, and when the approval lapses.
-
Dual Running
During a data platform migration, both pipelines process the same inputs and a comparison reports what differs. What makes the comparison trustworthy is the shared input, the key used to match rows, and a window long enough to see rare events.
-
Operational Subledger
Keep detailed operational movements traceable to source events, reconcile balances, and distinguish a signed movement table from double-entry journals.
-
Balance Roll-Forward
Explain a closing balance from its opening value and movements, align scope and cutoff, and detect errors hidden by matching aggregate totals.
-
Currency Minor Unit
Store money with its currency and scale, check provider-specific representations, and convert amounts without assuming that every currency uses cents.
-
Payment Authorization and Capture
Separate a payment hold from capture, keep expiry and partial capture rules explicit, and avoid treating either event as a bank deposit.
-
Integrating Payment, Commerce, and Finance Data
Trace payment movements across orders, wallets, processor balances, and finance reports while preserving currency, timing, and correction history.
-
Database Ingestion and Change Data Capture
Build database copies with explicit snapshot boundaries, safe progress tracking, delete handling, and reconciliation.
-
Reconciliation
Learn how reconciliation compares data with a reference and why matching row counts or totals can hide errors.