Deadline Propagation

Deadline propagation passes the time remaining for a request down every call it makes, so each layer can limit its own work to what the original caller is still willing to wait for. It replaces a set of unrelated per-layer timeouts with one shared budget.

The problem with independent timeouts

Suppose a user request goes through a gateway, a service and a model API. If each layer sets its own timeout, the layers do not know about each other. Hypothetical settings: each attempt gets a 10 s timeout and each layer makes up to 2 attempts, and the model API hangs.

• The service’s two attempts against the model take 20 s.
• The gateway gives up on the service after 10 s and tries again, so the service is asked twice.
• The caller waits 20 s for a failure, and the downstream layers do up to 40 s of work that nobody is waiting for.

None of the timeouts is wrong in isolation. The problem is that the sum of the layers’ waits is unrelated to what the user can tolerate, and work continues after the result is useless.

Mechanism

Fix an absolute deadline when the request enters the system: \(D = t_{\text{start}} + \text{budget}\). Every call carries it. Before starting an operation, a layer computes the remaining time \(r = D – t_{\text{now}}\). If \(r \le 0\), it fails immediately without making the call. Otherwise it sets the operation’s timeout to

\[ \text{timeout} = \min(\text{usual timeout},\ r), \]

and passes the same deadline on to the calls it makes.

With a 12 s user deadline and a 10 s attempt timeout, the first attempt may use 10 s, and the second gets \(\min(10, 12-10) = 2\) s. The total wait is exactly 12 s, with 2 attempts. The second attempt is short, and a layer that knows it can decide whether 2 s is worth attempting at all: if the typical call needs 4 s, it is better to stop and take the fallback.

Deadline versus timeout across a network

A timeout is a duration; a deadline is a point in time. Passing an absolute time between machines depends on their clocks agreeing. gRPC avoids this: it describes deadlines as the point past which the client is unwilling to wait, and when a call is forwarded it converts the deadline to a timeout with the elapsed time already deducted. This way, each hop receives a remaining duration rather than a timestamp. Some gRPC implementations propagate the deadline automatically to nested calls, and others require it to be turned on, so check the behavior of the library in use. Within one process, a monotonic clock is preferable to wall-clock time.

Where it applies

• Retry loops: a retry is allowed only when the backoff delay plus a minimum useful attempt time still fits before the deadline. Otherwise the loop stops and returns the best available result.
• Agent and workflow loops: a bounded run uses the remaining time as the timeout of the next model or tool call, so that a slow tool cannot run past the end of the run.
• Queues: a request that has already passed its deadline while waiting should be dropped when dequeued rather than processed.
• Cancellation: a deadline is most useful when the downstream operation really stops when it expires. If it only stops waiting, the work continues and still consumes resources.

Conditions and cautions

• A propagated deadline needs a budget that makes sense end to end. Divide it deliberately: if the first of three steps can use all of it, the later steps are starved. Some systems reserve a slice for the final stage, such as formatting the response or the fallback.
• Time spent waiting for a person (an approval, a user reply) is a different clock. Counting it in a machine-call deadline expires every request that waits.
• Deadlines bound latency, not load. Combine them with a retry budget and a circuit breaker so that failed work does not multiply.

References

• gRPC, Deadlines