Change Data Capture (CDC)
Change Data Capture, or CDC, identifies and exposes changes to source data so that other systems can consume them. Database CDC commonly captures inserts, updates, and deletes. Log-based tools read database change logs; other approaches may use triggers or queries, with different coverage and costs.
A change is different from a current row
An order changes from pending to paid and then to refunded. A current-state query may show only refunded. Captured change records can preserve the transitions if the capture method observes them and the destination retains them. Debezium, for example, uses database-specific mechanisms such as MySQL’s binlog and PostgreSQL’s logical replication stream.
A new downstream copy often starts with a snapshot, a view of existing rows, then applies changes. The handoff must avoid a gap or uncontrolled overlap. A snapshot taken today cannot recover every transition that happened before capture began.
Capturing changes is not the whole solution
If a consumer always overwrites each order with its latest state, it still has no queryable history of earlier states. History requires an appropriate storage design and retention policy.
Consumers must also decide how to apply deletions, handle repeated delivery, and prevent older updates from overwriting newer ones. CDC does not automatically guarantee instant arrival or exactly one effect at every destination. Its value is access to changes; reliable interpretation and application remain part of the pipeline.
Reference: Debezium: Architecture.
Discover more from Insightful Data Lab
Subscribe to get the latest posts sent to your email.
