Fencing Token

A fencing token identifies an ordered writer generation. A protected resource checks it when applying a change so an obsolete worker cannot overwrite work after ownership has moved on.

A worker with generation 7 may pause while a replacement obtains generation 8. When the old worker resumes, the target must reject generation 7. A lease expiring in a scheduler does not by itself stop the old process from writing.

Issuance must come from an authorized ordering mechanism, and validation must be serialized with the protected mutation. Every relevant write path must enforce the rule. A token copied into a log field without a target check provides no fencing.

Fencing does not prove that the newest worker has the newest source data, nor does it deduplicate repeated requests by that worker. Combine it with input versions, operation identity, and publication rules. A single-database generation check illustrates the rule without implementing a distributed lease service.

See Building Incremental and Idempotent Pipelines for a transactional example.

Reference: Official documentation.


Discover more from Insightful Data Lab

Subscribe to get the latest posts sent to your email.

Similar Posts

Questions, corrections, or additional insights?

This site uses Akismet to reduce spam. Learn how your comment data is processed.