Delivery Semantics

Delivery semantics describe how many times a record can reach its destination when things go wrong. Apache Kafka’s design documentation states the three standard guarantees: at most once — “messages may be lost but are never redelivered”; at least once — “messages are never lost but may be redelivered”; and exactly once — “each message is processed once and only once.” Descriptions here follow Kafka documentation checked in September 2026.

Why the middle one is the common default

The root difficulty is that a sender cannot distinguish a request that was lost from a response that was lost. Kafka’s documentation compares this to inserting into a table with an autogenerated key: after a network error, the producer does not know whether the write committed. Retrying risks a duplicate; not retrying risks losing the record. Choosing which risk to take is what produces at-least-once or at-most-once.

The consumer side has the same shape, and the documentation lays it out as an ordering choice: save your position and then process, and a crash in between means the work never happened (at most once); process and then save your position, and a crash in between means the work happens again (at least once). There is no ordering that avoids both, which is why at-least-once with harmless repetition is the usual target.

What “exactly once” requires

Kafka’s documentation is unusually direct about the marketing problem: many systems claim exactly-once delivery, “but it is important to read the fine print, because sometimes these claims are misleading” — specifically where producers or consumers can fail, where there are multiple consumer processes, or where data written to disk can be lost.

What makes the guarantee real is that the record’s effect and the record of progress are committed together. Inside Kafka this is achievable: an idempotent producer lets the broker discard resends using a producer ID and sequence number, transactions write to several partitions atomically, and the consumer’s offset can be written in the same transaction as the output, with readers at the read_committed isolation level seeing only committed results. Writing to an outside system, the documentation notes, runs into “the need to coordinate the consumer’s position with what is actually stored as output,” and states that exactly-once delivery to other destinations “generally requires cooperation with such systems.” The simpler route it recommends over a two-phase commit is to store the offset in the same place as the output.

Processing frameworks describe the same dependency. Spark’s Structured Streaming guide says that “using replayable sources and idempotent sinks, Structured Streaming can ensure end-to-end exactly-once semantics under any failure.” Flink’s documentation states that exactly-once state updates require the source to participate in its snapshotting mechanism, and that end-to-end exactly-once delivery additionally requires the sink to take part in checkpointing — its table of bundled sinks lists several as at-least-once only.

Reading a claim

The practical conclusion is that exactly-once is a property of a whole path rather than a feature of one component. A pipeline can claim it only if the source is replayable, the output is durable before the position advances, and one of two things closes the remaining duplicate window: the position committed atomically with the output, or a destination where re-applying the same records changes nothing. Those two are alternatives, not a pair.

What to say when a link is missing depends on which link, and the distinction is the useful part.

• The source cannot be replayed. Then at-least-once is not available: whatever was in flight at the failure is simply gone. At-most-once can still be an accurate description, provided the path does not duplicate. The honest options are to durably record arrivals before acknowledging them, to obtain a re-delivery commitment from the sender, or to state the possible loss and its bound. Calling this at-least-once is the error that hides a gap.
• Replay works, the output is durable before the position is recorded. Then duplicates are possible and loss is not, which is what at-least-once actually describes — and idempotent application at the destination is what makes the repeat harmless.
• Replay works, but the position is recorded first. Then loss is still possible even though the input is retained: a crash after the position advances and before the output lands leaves a normal restart reading past records nobody wrote. Replayability does not save you here, because nothing knows to go back. This is the case most often mislabelled at-least-once.
• Neither replay nor ordered durability. Both duplicates and loss are possible, so there is no guarantee to name. Reserve at-most-once for a path that can lose records but not duplicate them — usually one that acknowledges on receipt and never retries — rather than using it for “something may go wrong in both directions.”

Three separate things are doing work in that list, and conflating them is what produces the confident wrong label. Ordering — output durable first, position second — decides whether a crash loses records or repeats them. Atomic commit of the output and the position together removes the window entirely, and is available only when both live somewhere that can commit them as one. Idempotence does not remove any window; it makes the repeat that ordering leaves you with harmless. So a destination transaction around the write alone is not enough: unless the position is inside it, reprocessing still duplicates, and the transaction only guarantees each attempt is all-or-nothing.

Which means atomic commit and idempotent application are alternative ways to close the duplicate path, not a pair to demand together, and which is available is a property of the destination. How that plays out in a working pipeline, and what to check when the numbers disagree anyway, is worked through in The Job Succeeded and the Numbers Are Wrong.

References: Apache Kafka documentation, Design: Message Delivery Semantics; Apache Spark, Structured Streaming: Fault Tolerance Semantics; Apache Flink, Fault Tolerance Guarantees of Data Sources and Sinks.