Load Shedding

Load shedding is the deliberate rejection of some requests during overload so that the requests that are accepted can be served. It is a choice to fail part of the traffic quickly rather than all of it slowly.

The reasoning rests on what happens without it. A service accepting more work than it can complete does not degrade gracefully: queues grow, latency rises past the point where callers have already timed out, and the system spends its remaining capacity producing responses nobody is waiting for. Google’s SRE material describes this and the associated failure modes, and the practical conclusion is that serving a reduced share of traffic well is strictly better than serving none of it.

Rejection has to be cheap to be useful. If refusing a request costs as much as serving one, shedding provides no relief — which is why the check belongs as early in the path as possible, before expensive work such as authentication against a remote store or a database query.

The response should tell the caller why work was refused. RFC 6585 defines HTTP 429 for requests exceeding a rate limit; RFC 9110 defines 503 for temporary service unavailability, including overload. Either may include Retry-After, but the header does not itself prevent retries. Clients must honor the guidance and use suitable backoff, jitter, and retry budgets, while the server still needs admission controls for callers that do not cooperate.

The design decision that remains is what to shed, and it depends on business priorities and on what the system can enforce. Rejecting uniformly is simple and treats a checkout attempt like a background report refresh. Prioritizing requires knowing which traffic matters, which means someone has to have decided in advance. It also requires that the rejecting service can identify that priority from a signal it trusts, such as a criticality label set near the edge and propagated with each request, as Google’s SRE material describes. Rejecting low-priority requests helps only if it frees the resource that is actually constrained, and only if the check can run before the expensive work.

How it relates to retries and cascading failure is worked through in Partial Failure, Timeouts, and Retries.

References

• Google, Site Reliability Engineering, Handling Overload
• RFC 6585, Additional HTTP Status Codes
• RFC 9110, 503 Service Unavailable