Persistent Sending Queue

A persistent sending queue stores pending outgoing records on durable storage so supported restarts can resume delivery. It protects only the data accepted and persisted under its storage rules. It does not automatically protect earlier in-memory processing or the original source.

If an upstream batcher acknowledges a record while retaining it only in memory, a crash before enqueue can lose it despite the downstream disk queue. Trace acknowledgement and checkpoint boundaries through the entire pipeline. Queue storage and file-read checkpoints serve different purposes even when they share one storage extension.

Bound disk use and specify full-queue and retry-exhaustion behavior. A lost acknowledgement can still cause replay, while disk failure, expired retention, or permanent rejection can cause loss. Capacity helps survive an outage only if the source remains recoverable and recovered output capacity can drain the backlog. Test actual component versions rather than inferring guarantees from the word persistent.

Reference: Log and telemetry collection lab.