Request Criticality

Request criticality is a label carried by a request that says how much its failure matters, so that overload protection can drop the least important work first. It turns we are out of capacity from an accident into a policy.

The scheme described in Google’s SRE book uses four values, and the definitions are worth having exactly because each one states an expectation rather than a priority number.

Value	Definition
CRITICAL_PLUS	“Reserved for the most critical requests, those that will result in serious user-visible impact if they fail”
CRITICAL	“The default value for requests sent from production jobs. These requests will result in user-visible impact, but the impact may be less severe”
SHEDDABLE_PLUS	“Traffic for which partial unavailability is expected. This is the default for batch jobs, which can retry requests minutes or even hours later”
SHEDDABLE	“Traffic for which frequent partial unavailability and occasional full unavailability is expected“

These names describe Google’s RPC convention, not an industry standard. Its services are expected to provision for all anticipated CRITICAL and CRITICAL_PLUS traffic. Lower levels accept weaker availability; that does not remove their demand, deadlines, or recovery work from capacity planning. Distinguish protected demand from best-effort demand and state how both use shared capacity.

Four levels, and why not more

Google reports that four values were sufficient for almost every service it examined. More values could classify work more finely, but every criticality-aware system would need additional configuration and operating effort.

A level affects thresholds, quotas, dashboards, alerts, and rejection logic across services. Add one when it changes a meaningful allocation decision and the organization can operate it consistently. Four is the source’s workable convention, not a universal limit on useful distinctions.

Defaults also carry policy. In this scheme, production jobs default to CRITICAL while batch jobs default to SHEDDABLE_PLUS. Those choices reflect the expected impact and retry tolerance of those populations. Review uncategorized traffic explicitly; protecting every unlabeled request can itself crowd out work with a stronger claim.

It is not the same as urgency

The most common error is to collapse criticality into latency. The source is explicit: “the criticality of a request is orthogonal to its latency requirements and thus to the underlying network quality of service (QoS) used.”

Its example is the clearest case available. Search-as-you-type suggestions are “highly sheddable (if the system is overloaded, it’s acceptable to not display these results), but tend to have stringent latency requirements.” An answer that arrives late is worthless, and not answering at all is fine. Read as two independent axes:

• Importance decides whether the request is served when capacity is short.
• Urgency decides how long it is worth keeping, and therefore its deadline and its place in a queue.

A batch export shows why both axes need a third question: which unit is being judged. The job has to finish by its deadline; any single attempt within it can be refused, because a refusal costs a retry rather than an outcome. That is exactly why such traffic sits in the sheddable tier: shedding this call is cheap, and protecting every call of a job that has hours of slack would push out interactive requests that have none. A typeahead suggestion is unimportant at both levels: drop it freely, and give it a tight deadline so it never occupies a thread it cannot repay. Conflating the job with the call produces the two familiar bugs — protecting individual batch calls as if the job were due now, and queueing suggestions until they are stale. The export does need protection when its slack runs out: as the contractual deadline approaches, or when the retries themselves are what is failing, its criticality is a decision to revisit rather than a fixed property.

Set it at the edge, and let it propagate

A label is only useful if the component that eventually runs short of capacity can see it, and that component is usually several hops away from whoever knew what the request was for. Two mechanisms make that work.

Automatic inheritance. Google’s RPC system gives outgoing calls the incoming request’s criticality by default. That carries context to dependencies that would otherwise need to infer the caller’s purpose. Deliberate overrides remain necessary when a downstream operation has a different role.

Assign it where the business meaning is known. Google commonly sets criticality near browsers and mobile clients, in HTTP frontends. A trusted batch coordinator or business service can play the same role for other work. The relevant knowledge is impact, deadline, and recovery behavior, not simply whether a person is waiting.

A shared interpretation allows dependencies to apply the requested priority at depth. Propagating an enum is insufficient unless the rejecting services and client-side controls honor it consistently; validate that behavior across the path.

What it actually controls

Criticality earns its keep by being wired into the mechanisms that reject, not by being recorded. In the documented system it is “a first-class notion of our RPC system,” and four behaviors follow from it.

• Quota exhaustion drops from the bottom. “When a customer runs out of global quota, a backend task will only reject requests of a given criticality if it’s already rejecting all requests of all lower criticalities.” Strict ordering, not weighting.
• Local overload drops from the bottom, earlier. “When a task is itself overloaded, it will reject requests of lower criticalities sooner,” with “higher thresholds for higher criticalities” as utilization rises. One utilization signal, several trigger points.
• Client-side throttling accounts per level. “The adaptive throttling system also keeps separate stats for each criticality,” so a client that is being refused its batch traffic does not throttle its interactive traffic.
• Quotas can be set per level. Per-customer limits “can be set per criticality,” which is how a shared service lets a client have generous batch capacity and a small interactive allowance, or the reverse.

The third item is the one teams miss when they bolt a priority field onto an existing system. If your protection mechanisms average across levels, the label changes nothing under load — it has to be in the comparison that each mechanism makes.

A simpler version of the same idea appears in load-balancer configuration, and it is a reasonable place to start if you have no labels at all: drop “requests for low-priority operations like background synchronization, but keep serving interactive user sessions.” More selective schemes — “identifying clients to be more selective about what work is dropped, or picking requests that are more important and prioritizing” — are noted as “more likely to be needed for shared services,” which is the honest scoping. A single-tenant service with one traffic type does not need this.

Adopting it

Adoption takes four steps.

1. Name the levels in your own vocabulary, mapping each to an expectation rather than a number: what users experience when this class is dropped, and whether capacity is provisioned for it.
2. Set it at the entry point and carry it in whatever metadata your calls already propagate. If it cannot cross a service boundary, it will stop being true at the first hop.
3. Wire it into one mechanism first — usually the server’s own rejection threshold — so there is something observable before you touch quotas and clients.
4. Report rejections by level. Shedding the bottom level is expected only within its agreed loss and deadline limits; exceeding those limits or shedding protected work requires investigation. Without the breakdown, both appear as an error rate.

And the reason to bother, which is worth stating plainly: a system without criticality still decides what to drop. It just decides during the incident, by whichever component reaches its threshold first — which means the work that is dropped is the work that happened to arrive at the busiest place, not the work that mattered least. How the surrounding mechanisms use the label, and what to do instead of dropping, is worked through in The Shedding Region Looked Healthiest; the act of dropping itself is load shedding, and serving cheaper answers instead is graceful degradation.

References

• “Handling Overload,” Google SRE Book
• Google SRE Book — “Addressing Cascading Failures”

The four value names are one company’s RPC convention, not an industry standard.