Error Budget

An error budget is the permitted failure allowance under a service objective. For a good-event ratio with target s expressed as a fraction, the allowed failure fraction is 1 − s. With N eligible events and E failures in the stated window, allowed failures are (1 − s) × N, and remaining allowance is (1 − s) × N − E. At 99.99%, the permitted fraction is 0.01%. A negative remaining allowance indicates a missed objective.

The Site Reliability Workbook applies this arithmetic to ratio indicators. A latency value in milliseconds or throughput in requests per second needs an appropriate good-event definition before the same formula can be used. Define the population, window, and measurement boundary, including no-request and missing-telemetry handling. The calculated allowance can exist without a policy; a policy determines how the organization uses it.

What it is for

The SRE introduction presents error budgets as a way to align development’s interest in change with operations’ responsibility for stability. Both groups can evaluate a proposed release against an agreed reliability commitment instead of arguing from incompatible goals.

Remaining budget permits risk under the release policy; it does not waive testing or other release criteria. When the allowance is threatened or exhausted, the agreed response can slow selected releases, pause them, or prioritize reliability work. The policy must identify the changes it restricts and the exceptions it allows.

The shared measure helps the teams compare impact and decide how to use their engineering time:

  • Common decision evidence. Developers and operators assess reliability against the same stated target and counting rules.
  • Incident comparison. At 99.9% over three million eligible requests in four weeks, the allowance is 3,000 failures. An incident causing 1,500 consumes half of that allowance. This measures impact under the selected indicator; equal counts need not mean equal business harm.
  • Earlier risk reduction. Developers have a reason to improve testing or rollout safety as the allowance shrinks, before another incident forces a pause. That incentive depends on a credible operating agreement.
  • Target review. Persistent release constraints are a reason to examine the objective, current design, and resources. Relaxing the objective may be an option when user needs permit it, but it is not the only response.

The SRE book’s aim is useful innovation within accepted risk. Phased rollouts and experiments with limited exposure, including the book’s 1% example, can reduce the impact of a bad change. Failures remain harmful to affected users even when they fit within the allowance; the budget makes the risk explicit rather than making outages desirable.

Note the corollary that trips teams up: an unspent budget is not automatically a win. A service that ends every quarter far inside its budget may have been buying reliability nobody asked for with velocity it could have had — or it may have had a quiet quarter, or it may have got both from safer release machinery, which is the outcome the whole approach is meant to produce. So treat a persistently unused budget as a reason to re-examine the target and the risks being declined, and check the opportunity cost rather than assuming it: is there a change the team deferred, and would shipping it actually have spent the budget? Neither spending the budget nor manufacturing failure is the goal.

Make the policy actionable

The workbook calls for stakeholder approval of the SLO, operational confidence that it is feasible, commitment to using the budget in decisions, and a process for refining the objective. Record these commitments in a written policy so the calculation has an agreed response.

Without that agreement, teams can still report compliance and calculate the allowance. They may lack the authority or shared priorities needed to act on it.

Name the actions and their owners. The workbook describes responses such as these; they are choices to define locally, not a mandatory escalation ladder.

ResponseWhat it commits
PrioritizeGive recent reliability defects priority over other work according to the agreed trigger.
Focus engineering effortAssign the team to reliability improvements, with authority to defer conflicting feature requests.
Restrict releasesPause the specified changes until the policy’s resumption conditions are met, with an explicit exception process.

Authority to defer conflicting demands makes the commitment practical. The workbook’s example policy permits P0 and security fixes during its freeze. A local policy should state the approval path for emergency recovery and security changes, along with release-resumption criteria. Remaining budget alone does not authorize a release that fails other checks.

Approval also tests whether the plan is credible. Excessive toil may indicate an unrealistic target, inadequate resources, avoidable design problems, or a misleading indicator. If users suffer before the policy prompts action, coverage or thresholds may need improvement. Consider more evidence, better design, resources, and revised objectives together. Lowering an objective also changes when its response is triggered, so product owners must assess the resulting user impact.

Keep measurement and enforcement credible

  • Assign enforceable authority. A release pause or a change in priorities needs an owner empowered to carry it out, with a route for resolving competing demands. An advisory budget can inform decisions, but it does not itself guarantee action.
  • Resolve disagreement explicitly. New conditions or measurement errors can justify revisiting an approved policy. Use the agreed escalation path, record the reason and effective date, and distinguish legitimate corrections from retroactive changes made to improve the score. Reconsideration alone does not prove the original agreement was insincere.
  • Count the declared user impact. Infrastructure and dependency failures can consume budget even when this team did not cause them. Assigning remediation and deciding whether other teams continue feature work are separate policy decisions. The workbook’s example distinguishes cause-dependent feature work from release restrictions; it does not erase user impact because another team owns the cause.
  • Observe the budget during the window. Timely warnings can support action before exhaustion. An end-of-quarter report can inform later planning, but cannot prevent the failures that have already occurred.

The window changes the calculation, not the physical state of the service. A calendar boundary starts a new measurement period; it does not repair a fault or establish the future request count. In a rolling window, both eligible requests and failures enter and leave. Recovery need not be smooth or wait for failures to age out. For example, at a 99% target, 100 requests with two failures leave an allowance of −1 failure. If 200 successful requests enter before any old events leave, 300 requests with the same two failures leave +1. A concentrated set of failures aging out can also move the ratio sharply. Specify what operational evidence is required for normal releases to resume, separately from statistical compliance.

Adopting one

Agree an objective that users can accept and operators can defend. Define its measurement, derive the allowance, and approve actions, owners, exceptions, and escalation before relying on the policy. Set up timely monitoring and review the assumptions as the service changes.

The shared policy can make release decisions and incident discussions less contentious because the parties already know which evidence and actions apply. Check whether it changes decisions in practice. SLIs, SLOs, and Error Budgets: Choosing Reliability Targets explains target selection; risk tolerance covers the judgment behind the number.

References


Discover more from Insightful Data Lab

Subscribe to get the latest posts sent to your email.

Similar Posts

Questions, corrections, or additional insights?

This site uses Akismet to reduce spam. Learn how your comment data is processed.