Change Data Capture (CDC)

Change Data Capture, or CDC, identifies and exposes changes to source data so that other systems can consume them. Database CDC commonly captures inserts, updates, and deletes. Log-based tools read database change logs; other approaches may use triggers or queries, with different coverage and costs. A query-based extraction keyed on an update timestamp — an extraction watermark — cannot normally see hard deletes and cannot guarantee capture of every intermediate state, which is the usual reason teams move to the log.

A change is different from a current row

An order changes from pending to paid and then to refunded. A current-state query may show only refunded. Captured change records can preserve the transitions if the capture method observes them and the destination retains them. Debezium, for example, uses database-specific mechanisms such as MySQL’s binlog and PostgreSQL’s logical replication stream.

A Debezium PostgreSQL change event identifies the operation and its source, and can include before and after values. Which values are present depends on the operation and replica identity configuration. With the default replica identity, the previous image of an updated row normally contains only its key, so it cannot identify every changed non-key column by comparison. Unchanged TOAST values can be represented by an unavailable-value placeholder rather than the stored value.

Source metadata supplies positions for tracking progress, but comparing updates requires a defined scope: the same source stream, transaction boundaries, and any needed within-transaction order. An LSN alone is not a universal version across databases or independently routed partitions.

The three capture methods differ in exactly those terms, which is why the choice is usually settled before the tool is chosen.

Method	Sees	Cost on the source
Query by changed-at column	Latest state of selected rows; normally misses hard deletes and can miss intermediate states	A repeated scan or index range read on a live table
Triggers	The operations and tables the triggers are registered for, as each change happens inside its transaction. In PostgreSQL, row-level INSERT, UPDATE, and DELETE triggers do not see TRUNCATE, which needs its own statement-level trigger, and a trigger’s writes roll back with the transaction, so only committed changes should be passed on	Extra work inside each writing transaction, and schema objects to maintain
Database log and logical decoding	Captured changes within the configured scope. Physical WAL order is distinct from commit order; a logical protocol may send in-progress transaction blocks that require a later commit or abort decision. Row images depend on the log and connector settings	Additional logging, decoding, and retention can cost resources. PostgreSQL logical WAL records more information; full row images can increase write volume further

A new downstream copy often starts with a snapshot, a view of existing rows, then applies changes. The handoff must avoid a gap or uncontrolled overlap — see snapshot and stream handoff, and logical replication slot for what the source retains while a consumer is away. A snapshot taken today cannot recover every transition that happened before capture began.

Capturing changes is not the whole solution

If a consumer always overwrites each order with its latest state, it still has no queryable history of earlier states. History requires an appropriate storage design and retention policy.

Reading a database log couples the consumer to the captured table schema. If that schema is internal and its owner has made no compatibility commitment, a refactoring can break ingestion without changing the application API. This is a missing agreement, not an inevitable property of CDC. The source and consumer can agree on published fields, meanings, compatible changes, notification, and ownership as a data contract. A deliberate export or domain-event interface is another option, but it also needs a maintained contract. Where a source cannot be changed, record the coupling and the procedure for detecting and adapting to changes.

Consumers must decide how to apply deletions — a tombstone is a common carrier — and handle repeated delivery with idempotent writes or deduplication. For complete state replacement, atomically comparing a source version with the stored version can prevent an older message from overwriting newer state. The same discard rule is unsafe for partial changes. Starting with status=open and amount=100, applying v2={amount:80} and discarding v1={status:paid} loses the status change. Preserve which fields are supplied, distinguish an unavailable value from an explicit NULL, and apply the required changes in source order or use a merge contract that preserves their effects, such as appropriate per-field versions. Deletes also need version-aware handling so a late update cannot revive a deleted row. CDC supplies changes; the destination must define their interpretation and durable application.

References:

• Debezium: Architecture
• Debezium Documentation, Debezium connector for PostgreSQL
• PostgreSQL: Streaming of Large Transactions
• PostgreSQL: Write Ahead Log
• PostgreSQL: Overview of Trigger Behavior