Snapshot and Stream Handoff

Snapshot and stream handoff initializes a copy from a consistent view and then continues it with captured changes. The snapshot and starting change position must describe one coordinated boundary. Reading the table and independently recording the latest log position can leave a gap.

Suppose the snapshot sees order A as paid. A refund commits during the scan. The handoff must either include it in the snapshot’s view or retain it for later application, not silently exclude it from both. Use the engine or connector’s supported snapshot protocol and retain the stream while scanning.

How the boundary is actually created

“Use the supported protocol” is the right advice and it hides the mechanism, so here is one concrete instance of it. PostgreSQL ties the two together at slot creation: when a replication slot is created through the streaming replication interface, “a snapshot is exported … which will show exactly the state of the database after which all changes will be included in the change stream.”

The boundary becomes usable only if the client imports the exported snapshot while it is valid and retains the changes needed to catch up. For PostgreSQL’s exported-snapshot path, the connection and transaction sequence matters.

1. Create a logical slot with snapshot export through the replication protocol. Save the returned snapshot identifier and consistent point. Keep that connection open without issuing another command until the reading transaction has imported the snapshot.
2. On a separate SQL connection, begin a REPEATABLE READ transaction and run SET TRANSACTION SNAPSHOT before its first query or data-changing statement. A default READ COMMITTED transaction cannot import it. PostgreSQL permits certain serializable imports too, subject to additional conditions.
3. Read the tables within that transaction. Concurrent writes do not change its snapshot, but retained WAL, database cleanup, disk capacity, and configured time or retention limits constrain how long the scan can continue. Monitor those limits and restart with a new coordinated boundary if required history becomes unavailable.
4. Commit the completed scan, then consume from the slot’s returned consistent point with all required changes still available. The documented result is that the dumped state “can be updated using the slot’s contents without losing any changes.”

Compare that with the two ways it is commonly done wrong. Reading the table first and then creating the slot loses everything committed in between. Creating the slot first but reading without adopting the exported snapshot gives a read at some later, unknown point, so changes between the slot position and that read arrive a second time — survivable only if the destination is idempotent, and silently wrong if the apply logic assumes each change is new.

Debezium’s chunked incremental snapshot solves the same problem differently, and the two should not be mixed up. There the scan is split into chunks that run alongside the stream without a long-lived transaction, and overlaps between scanned rows and streamed changes are resolved per chunk — typically by bracketing each chunk between markers and dropping buffered rows whose keys the stream also touched. That is what makes it restartable and easier on the source; it is not the exported-snapshot boundary, and its correctness argument is the overlap rule rather than a single consistent view.

Publish only the complete validated snapshot, apply the required suffix with retry-safe ordering, and checkpoint durable results. Restarting a scan against new data requires a compatible new boundary. This handoff establishes capture continuity, not business correctness or permission to discard old history.

References:

• PostgreSQL: Logical Decoding Concepts
• PostgreSQL: Streaming Replication Protocol
• PostgreSQL: SET TRANSACTION
• Debezium PostgreSQL connector: Incremental snapshots