Throttling

Throttling is the deliberate limiting of how much a user, tenant, application, or workload may consume, by delaying or rejecting requests once usage passes a threshold. Microsoft’s Throttling pattern describes the goal as limiting the resources that an application instance, an individual tenant, or an entire service can consume, so that the system keeps functioning and meeting its service-level objectives under sudden or sustained load.

Why throttle instead of just scaling

Adding capacity takes time and money. If demand rises faster than new resources arrive, or beyond what the budget allows, the system can exhaust what it has and fail for everyone. Throttling protects the system while that happens. The pattern describes using it alongside autoscaling: when load crosses a soft limit, scaling starts, and throttling rejects the excess until the new capacity is online.

Common forms

• Per-principal limits — rejecting requests from a user or tenant that has exceeded its rate, which requires attributing every request to someone.
• Graceful degradation — switching off nonessential features so essential ones keep resources.
• Load leveling and deferral — queuing work, or postponing lower-priority work until load eases.
• Outbound limits — reducing your own calls to a dependency that is failing, instead of flooding it with retries.

An example: Microsoft Fabric capacities

Microsoft Fabric’s general throttling policy uses accumulated, smoothed capacity consumption and progresses through the stages below. Workload-specific behavior also matters; the general table does not describe every service identically.

Under the general policy, running operations can continue while new work is delayed or rejected. There are workload-specific exceptions: Eventstreams can reduce resources for running streams, and Real-Time Intelligence has different behavior at the initial delay stage. Throttling is assessed per capacity, but a workload elsewhere can still be affected if it depends on a throttled service. Separate capacities reduce one shared resource boundary; they do not remove application dependencies. See bursting and smoothing for how accumulated usage arises.

Doing it well

• Limit what saturates first. A requests-per-second cap can reduce in-flight work and calls to a dependency, but it does not by itself guarantee their limits. Longer request duration, bursts, or more downstream calls per request can increase pressure at the same admitted rate. Where those quantities constrain service, also limit in-flight work or admission to the particular dependency.
• Tell callers. Return HTTP 429 or 503 with a Retry-After header and enough context to back off deliberately. Clients must then honor it with backoff rather than immediate retries.
• Propagate overload. A service that hides a downstream throttle by retrying silently turns one overloaded component into a retry storm.
• Keep rejection cheap, and keep limits adjustable at runtime, because incidents are the worst time to deploy.

Throttling is closely related to load shedding and is a standard defense against the noisy neighbor problem. How Fabric’s version shapes platform design is worked through in Microsoft Fabric Architecture: OneLake, Capacity, and the Move from Synapse.

References

• Azure Architecture Center, Throttling pattern
• Microsoft Learn, Understand capacity throttling and smoothing