Durable Execution

Durable execution is a way of running multi-step code so that its progress is recorded outside the process, allowing it to resume after a crash, restart, or long wait reusing results for steps whose completion is durably recorded. External effects can still be repeated if a crash occurs before their completion record is saved. It is offered by workflow engines and orchestration frameworks, and it matters most for work that spans many calls to external systems or waits hours or days for something to happen, such as a person’s approval.

The problem it addresses is ordinary. A process that creates a return label, issues a refund, and sends a confirmation keeps its knowledge of which steps are done in memory. If it crashes after the second step, that knowledge is gone. Starting over blindly can repeat the label and refund. The application may be able to reconstruct progress from authoritative external records, but otherwise it needs reconciliation or a controlled pause rather than assuming nothing happened.

How it works

Microsoft’s Durable Functions documentation describes one common design. Orchestrator code coordinates the steps and automatically checkpoints its progress whenever it waits for one. After a restart, the orchestrator runs again from the beginning, but each step it asks for is looked up in the recorded execution history: if that step has a recorded completion, its stored result is returned instead of running it again, so the code reaches the point where it stopped with its local state rebuilt.

Replay imposes two rules. First, the coordinating code must be repeatable: the documentation states that orchestrators “must be deterministic”, because they replay multiple times and must reach the same decisions each time. Ordinary wall-clock reads, fresh random values, and direct external calls can break replay. Use the runtime’s replay-safe context time or ID APIs where available, or move non-deterministic work into activities with recorded results. Model calls and their selected actions need that boundary too. Version or migrate running workflows when code, prompts, or tool contracts change in ways that affect replay. Second, the steps with side effects are not guaranteed to run exactly once. The same documentation says “the runtime guarantees that each activity runs at least once during an orchestration”, and that a step which completed just before a crash, but before its result was recorded, may run again. It advises making step logic idempotent whenever possible.

What it does and does not provide

Durable execution provides a reliable record of progress and automatic resumption, including across long waits: a workflow can pause for days for an approval without holding a process open. It does not make external side effects safe to repeat. That still depends on each step being idempotent, usually by passing an idempotency key bound to a stable business-operation identity persisted before sending and reused under the receiver’s duplicate-protection contract. An attempt counter or model turn is not that identity; scope, argument matching, concurrency, and retention also matter. Nor does it undo completed steps when a later one fails permanently; a saga can organize an explicit recovery policy: choose compensation, forward recovery, or owned exception handling according to the business contract. A saga records that policy but does not make every effect reversible. Revalidate permissions and preconditions after long approval waits.

The idea resembles checkpointing in stream processing, where recorded state and replayed input restore a computation after failure, but it applies to individual workflow instances and their calls to other systems rather than to a continuous data stream. How it applies to model-driven agents that call tools is covered in The Refund Went Out Twice.

References

• Microsoft Learn, Durable orchestrations (Creative Commons Attribution 4.0).
• Microsoft Learn, Orchestrator function code constraints (Creative Commons Attribution 4.0).
• Microsoft Learn, Durable Task programming model (Creative Commons Attribution 4.0) — at-least-once activity execution.