Event Deduplication

Event deduplication recognizes repeated deliveries of the same logical event so that downstream processing can avoid counting it again. Generate a stable event ID when the event is created and preserve it on retry. Deduplicating equal-looking rows without an identity rule can erase legitimate repeated actions.

Two deliveries with one ID may differ in transport timestamps yet carry the same business payload. Compare the logical fields under a documented policy. If the same ID has two different amounts, quarantine or reject the conflict instead of keeping whichever arrived first.

Retain deduplication state for the supported retry and replay horizon, and coordinate its update with the intended effect. A check followed by an uncoordinated write can race. Two different event IDs for one order require a separate business-key rule; one order can also legitimately have several event types. Removing duplicates from a table does not by itself prevent repeated external emails.

Reference: Segment duplicate handling.