SLIs, SLOs, and Error Budgets

A leadership team proposes 99.99% availability across all services, up from a measured 99.9%. Under a time-based definition that counts full downtime over a 365-day year, that reduces the allowance from 525.6 minutes to 52.56 minutes. A request-based objective would instead limit failed requests. Before accepting either target, the team needs to identify what users would gain and which changes could deliver it. Extra redundancy might be expensive; removing a recurring fault or improving rollout automation might reduce both failures and operating work. The target alone does not determine the cost or release pace. This situation is illustrative.

Google’s SRE book describes an availability target as, “in a sense,” both a minimum and a maximum: it helps decide how much to invest in reliability and which useful changes to defer. Assess that opportunity cost before treating an above-target result as failure or evidence of overfunding. Safer engineering can exceed the objective without wasting resources, and a quiet quarter can produce the same result.

The book’s Chubby case shows how unusually reliable operation encouraged consumers to assume a dependency would never fail. Examine those expectations and failure handling before considering a deliberate reliability intervention; exceeding an objective alone is insufficient reason to make a service less reliable.

Why 100% is the wrong number

The Embracing Risk chapter evaluates reliability alongside features, cost, and user impact. A permanent promise of zero failures leaves no allowance for the residual risks that remain even after redundancy and careful operation.

The Site Reliability Workbook discusses why a 100% reliability objective is a poor operating target. The operating target needs to account for residual risk, the complete user path and the work required to improve the service.

Redundancy, health checks and failover reduce the residual risk but cannot eliminate simultaneous failures. The complete user path also includes devices, networks and other dependencies that can fail while the service works; assess each part’s contribution to the outcome.

Features, patches, hardware changes and scaling introduce some risk while improving the service. A zero-risk rule can prevent that necessary work, and a zero-failure allowance can turn any observed miss into a reactive priority shift instead of a considered choice between reliability work and other improvements.

The risk chapter and the SRE introduction use a 99% reliable smartphone to illustrate why users may gain little from raising a backend from 99.99% to 99.999%. That is an argument for checking marginal benefit, not a proof that the improvement is invisible. For a hypothetical independent serial path, 0.99 × 0.9999 = 0.989901, whereas 0.99 × 0.99999 = 0.9899901. The difference remains positive. Real benefit depends on who makes requests, failure correlation, traffic, and the consequences of a failed operation; this calculation is only a counterexample to a zero-benefit claim.

Cost also depends on the proposed implementation. The risk chapter warns that an increment of reliability may cost 100 times as much as the previous increment; it does not supply a universal multiplier. Additional replicas consume resources, and engineering spent on risk reduction cannot simultaneously build other features. But a simpler design or a fix for a recurring incident can improve reliability while reducing cost. Compare concrete options and their opportunity costs.

The resulting decision is how to provide reliability users need while preserving the capacity to improve and operate the service. More reliability can be valuable. The question is whether a particular improvement is worth its full cost.

From what users care about to a number

The indicator defines the measured outcome, the objective sets the acceptable level, the error budget quantifies the permitted shortfall, and the policy determines the response. Each depends on the definition before it.

A service level indicator measures an aspect of service delivered to users. The workbook recommends good events divided by eligible events as a useful form. With a nonzero denominator, 100% means that every included observation met that particular rule, not that every aspect of the service was healthy. Define which requests qualify, what makes one good, and how to handle no requests or missing telemetry. Neither absence of traffic nor a broken collector should silently become evidence of success.

Separate the SLI specification, the outcome that matters to users, from the SLI implementation, the way it is measured. For example, a specification about successful page loads can be implemented through logs, probes, or client instrumentation. Each implementation has different accuracy, coverage, and collection costs. Server logs cannot reveal every request that failed before reaching the server.

Choose a small set of indicators that covers the important user outcomes. Too many can obscure priorities, while too few can miss a failure users care about. Existing logs can be a practical first implementation when probes would take weeks, provided the unobserved paths are documented and later checked against incidents and user feedback.

A service level objective sets a target on an indicator over a defined window. The SRE book’s target-selection advice is useful when read together with the workbook’s guidance on iteration:

• Use current performance as evidence. Copying it without examining the effort required can lock in an unsustainable commitment. It can still provide an initial baseline when little else is known, provided the team revisits it.
• Keep the calculation understandable. Complicated aggregation can conceal which users or operations are failing.
• State bounded requirements. Specify the workload and service conditions instead of promising unlimited scale or permanent availability.
• Choose objectives that influence decisions. Each should help settle a real priority question while the set covers the important service outcomes.
• Refine the initial target. Avoid an unnecessarily strict commitment, but do not lower established user requirements merely to make the first SLO easy to meet.

An overly strict objective can consume disproportionate effort; a lax one can leave users with unacceptable service. An initial SLO therefore needs both a rationale and a review process.

Define what availability counts

Availability is often expressed as nines: 99.9%, 99.99%, or 99.999%. Each extra nine reduces the permitted failure fraction tenfold. For full downtime over a 365-day year, 99.99% permits 52.56 minutes; that conversion does not apply directly to a request-based measure.

Embracing Risk explains why Google used request success rate for the distributed services discussed there: some locations could continue serving while others failed. Its example is 2.5 million eligible requests in a day at a 99.99% daily objective, allowing 250 failed requests. The denominator and the period are part of that promise.

A process liveness check alone misses many forms of partial failure. A time-based indicator can nevertheless capture degradation if a good interval requires a critical function or a specified success ratio, with appropriate user or regional scope. Equal intervals weight time equally; a request-based measure weights periods by their request volume. Choose the weighting that represents the promised service.

The risk chapter reports ISP background error rates between 0.01% and 1% from the time of its writing. Those historical measurements offer context for its marginal-benefit argument, not a current benchmark or a ceiling on measurable service improvement. Check the actual users’ connection conditions and business outcomes before deciding that another improvement would have little value.

The budget is the remainder, and the policy is the point

For a good-event ratio with target s, expressed as a fraction, the allowed failure fraction is 1 − s. Over a defined window containing N eligible events and E failures, the error budget allows (1 − s) × N failures. The remaining allowance is (1 − s) × N − E. A negative result means the objective was missed. This arithmetic does not apply directly to an SLI expressed in milliseconds or requests per second; first define a corresponding good-event rule if a ratio budget is needed.

The workbook’s example uses a 99.9% success objective and three million eligible requests over four weeks. The allowance is 3,000 failures, and an incident responsible for 1,500 consumes half of it. That compares impact under this indicator; two incidents with equal failed-request counts can still have different consequences if they affect different operations or users.

The SRE introduction uses error budgets to align product development and service operation around an agreed reliability commitment. Remaining budget permits planned risk under the release policy; it does not waive testing, security, or other release criteria. When the budget is depleted or threatened, the policy can prioritize reliability fixes, slow selected releases, or pause them.

The incentive is to preserve room for useful changes. As the remaining allowance shrinks, developers have a reason to invest in safer releases and testing before further incidents force a pause. Phased rollout and limited experiments can reduce the impact of a bad change. Neither using all the allowance nor creating failures is a success criterion: safer delivery may leave budget unused while still increasing useful output.

The people enforcing the policy need authority to carry out its actions. Measurement also needs to remain about the declared user experience: a dependency or infrastructure failure can consume budget even when the service team did not cause it. Which team should fix the cause, and whether others should stop feature work, are separate decisions about responsibility and available remedies.

A budget can be calculated without a written policy, but that alone does not establish a response. The workbook calls for stakeholders to approve the SLO, operators to judge it feasible, the organization to commit to using it in decisions, and a process for revising it. The policy records how those commitments are enforced.

Specify the actions, owners, release exceptions, and exit criteria. A team assigned to reliability work needs approval to defer conflicting requests. The workbook’s example policy exempts P0 and security fixes from its release freeze and distinguishes release restrictions from cause-dependent allocation of engineering work. Use it as an example, then agree local approval paths for urgent recovery and security changes. A return within SLO need not prove that the underlying fault has been repaired; the policy should state what else must be true before normal releases resume.

When consumers rely on more than the promise

A service can satisfy its SLO while consumers depend on behavior it never promised. The Chubby example in Service Level Objectives describes this mismatch.

Chubby is Google’s distributed lock service. The book reports that outages of its global instance were so rare that consumers added dependencies assuming it would always be available. When it did fail, those consumers could not function appropriately, and some resulting outages reached users.

The response described in the book was to synthesize a controlled outage in a quarter when actual failure had not brought availability down to the intended level. Its purpose was to expose unreasonable dependencies soon after they were introduced. This is a documented practice in that case, not a claim about Google’s current operations or a general requirement to exhaust a budget.

The same chapter discusses occasional outages, request throttling, and avoiding unexpectedly fast behavior under light load as ways to manage overdependence. Each deliberately affects service and needs a justified scope. High observed reliability alone does not establish that one of these interventions is needed.

First inspect what consumers assume and how they handle failure. A dependency contract states what they can plan against; tests and incident evidence show whether they can degrade or recover. Unhandled dependency failures can propagate, including through the overload feedback described in cascading failure. Publishing a target and holding an internal margin help set expectations, but neither substitutes for that evidence.

A tighter internal SLO can provide operating margin before an externally promised level is missed. The numerical comparison assumes compatible populations, success rules, measurement boundaries, windows, and exclusions. If an internal rolling-window measure and a contractual calendar-month measure differ, calculate both; subtracting their headline percentages does not establish the margin.

A service level agreement includes consequences attached to meeting or missing a service commitment with users. They need not be financial, but they must belong to that agreement. General reputational damage or lost revenue does not itself create an SLA. The book uses public Google Search as an example of business consequences despite the absence of a public SLA. An internal error-budget response can likewise exist without becoming an agreement with customers.

Publishing the objective helps consumers judge whether the service fits their needs. It can discourage both reliance on guarantees the provider has not made and unnecessary avoidance of a service consumers believe is less reliable than it is. Revisit that communication when observed performance and expectations diverge.

Services whose customers want opposite things

Infrastructure services can have several client groups without a dedicated product owner. Embracing Risk describes engineers taking on the product decision, sometimes implicitly. Someone still needs the authority to reconcile the groups’ needs and accept the resulting tradeoffs.

The book’s storage example contrasts low-latency users, who benefit from nearly empty request queues, with offline analysis users, who prefer sustained work that keeps resources busy. At the operating points described, optimizing utilization conflicts with minimizing waiting. That is a concrete workload conflict, not a property of every shared service.

The documented response was to partition the infrastructure into service levels. Throughput-oriented clusters could cost as little as 10–50% of the low-latency clusters in that example. That figure describes the source’s case, not an expected saving for another platform. Distinct service levels and prices gave clients a way to evaluate the tradeoff.

A multi-tenant platform may still meet its users’ needs with one objective. Where it cannot do so economically, consider workload-specific objectives, reservations, priorities, or separate pools according to the resource that needs protection. Published tiers can make risk tolerance explicit, but a price choice alone does not prove that a tier is suitable. Check the client’s requirements, budget constraints, and remaining dependencies.

Turning business goals into a number

Business goals need an engineering interpretation before they become a defensible objective. Product owners assess user and business impact; engineers explain feasible designs, measurement coverage, operating effort, and cost. Neither group can settle the target well without the other.

The risk chapter starts with service expectations, effects on the provider’s and customers’ revenue, whether the service is paid, competing offerings, and whether customers are consumers or enterprises. These questions establish what a failure would mean. Technical evidence then helps estimate which improvements would change those outcomes.

The book illustrates the revenue calculation with requests of equal value. Under its simple linear model, moving from 99.9% to 99.99% adds 0.09 percentage points, and a \$1 million revenue base gives \$1,000,000 × 0.0009 = \$900 of additional revenue over the same comparison period and traffic base. An improvement costing less than \$900 passes that model’s test. For an actual investment, examine recoverable sales, retries, margins, churn, compensation, and whether failures concentrate in valuable transactions. Compare one-time investment and recurring operating costs over an explicit horizon; the example does not establish the return for every service.

The workbook asks product managers to judge the service adequate for users, developers to accept risk-reduction work when the budget is exhausted, and operators to judge the objective sustainable without excessive toil or burnout. The commitment includes the response to a miss, not only the reported percentage.

If the parties cannot agree, investigate why. The target may be wrong, but the measurement may also miss user harm, or the design and resources may be inadequate for a necessary target. More evidence, a design improvement, additional capacity, or a staged objective may be appropriate. A provisional target must respect existing commitments. An empowered owner accepts the business tradeoff with engineering input; an operations team can hold that role when it has the relevant authority.

Record authors, technical reviewers, business approvers, approval and review dates, the objectives, SLI implementations, calculation rules, and the reason for choosing them. Include the policy’s actions and escalation path. New conditions or measurement errors can justify revision; document the reason and effective date instead of silently changing definitions to improve past results.

Speed and stability are not opposites

A local decision can trade some release time for lower immediate risk without making slow delivery a general source of reliability. The longer-term question is whether engineering improvements allow the service to change both safely and quickly.

DORA’s January 2026 guidance reports that delivery speed and stability tend to improve together across its five metrics. That association does not prove that every review or gate is costless. Evaluate the risk it reduces and the delay it adds. Small batches, a rehearsed rollback, and canary releases can support safer change by limiting blast radius and improving recovery. Their usefulness depends on compatible recovery targets, representative observations, and effective stop conditions.

DORA also warns that broad numerical goals can encourage gaming. An SLO tied to rewards can create pressure to exclude failures or narrow the population. Preserve agreed definitions, investigate unexpected changes in counts, and use user outcomes alongside the score. Read delivery metrics and service objectives together while retaining their different event populations; neither set alone establishes that users received something valuable.

Starting from nothing

The workbook’s advice provides a starting sequence from user requirements through measurement to an agreed policy. Adapt that sequence to the service’s requirements and current evidence.

1. Choose a service and identify its users. Include the outcomes they depend on and existing commitments.
2. Describe the service outcomes before choosing instrumentation. Availability and latency are common starting points, but data services may need freshness or correctness measures.
3. Implement an affordable first measure. Existing logs may be sufficient to start; record the paths they miss and check coverage against incidents and user feedback.
4. Agree a target and its rationale. Use an empowered owner, assess feasibility with operators, and plan improvement where current performance falls short of user needs.
5. Approve the error budget policy before relying on it. Define actions, owners, exceptions, escalation, and release-resumption conditions. Resolve disagreements through evidence, design, resources, or revised objectives as appropriate.

Review the objective when users, workloads, failure impact, or operating conditions change. Investigate both persistent misses and a large unused budget, but do not infer waste from high availability alone. Consumer failure handling and the opportunity cost of deferred work need their own evidence.

For the leadership team proposing 99.99%, the useful next step is a service-specific decision: which user outcomes would improve, which design would deliver that improvement, what it would cost, and what the organization would do after a miss. A defensible target connects those answers to a measurable commitment and an operating policy.

References

• Marc Alvidrez, Embracing Risk — Site Reliability Engineering
• Chris Jones, John Wilkes, Niall Murphy, and Cody Smith, Service Level Objectives — Site Reliability Engineering
• Benjamin Treynor Sloss, Introduction — Site Reliability Engineering
• Google, Implementing SLOs — The Site Reliability Workbook
• Steven Thurgood, Example Error Budget Policy — The Site Reliability Workbook
• Nathen Harvey, DORA’s software delivery performance metrics — January 5, 2026