Provisioned Throughput

Provisioned throughput reserves processing capacity for a hosted generative model under a fixed-cost, fixed-term subscription. Google Cloud requires an eligible model and supported location when placing the order. The reservation’s units and service conditions depend on the product; it does not necessarily give the customer a physically exclusive server. Other providers offer capacity commitments with their own units and contracts.

Shared capacity and its limits

Pay-as-you-go model calls commonly draw from shared capacity without a term commitment. Google Cloud Standard PayGo assigns an organization-level baseline for supported usage tiers and serves excess demand on a best-effort basis; this differs from the Gemini Developer API’s project/model quotas. A 429 can reflect temporary resource contention or an applicable usage limit. Diagnose the cause before retrying: transient contention may warrant bounded backoff with jitter, whereas a daily limit may require reset, a lower request rate, or an approved increase. Set an overall deadline and retry budget as well as a wait cap. These controls complement throttling.

Google Cloud’s Error code 429 documentation distinguishes Standard from Single Zone Provisioned Throughput. Below the purchased amount, capacity errors that would otherwise be 429 are reported as 5xx; for Standard they count toward the SLA error rate, while the documented Single Zone case does not. This is a rule about those capacity errors and the relevant contract, not every possible server error. Eligible overages can be served and billed as PayGo under the request policy. Reserved capacity still needs application timeouts and a recovery plan.

Deciding how much to reserve

The reservation is paid for even when idle, so compare utilization and the service target with current prices. Raw token totals are not a portable sizing unit: Google’s model tables specify generative AI scale units (GSUs), throughput per second, and conversion rates for input, output, and cached tokens. As a simple arithmetic illustration, with an input weight of one and output weight of five, 100,000 input plus 100,000 output tokens consume 600,000 weighted units, not 200,000. This calculation alone does not size an order. Use the selected model’s actual rates, traffic distribution, burst timing, and latency measurements. Reserve an appropriate steady load and decide explicitly whether eligible online peaks may use PayGo or must be limited. Asynchronous batch prediction is a separate path and is not supported by this Provisioned Throughput reservation.

The fee commitment and the model assignment are different constraints. Google Cloud permits conditional changes to an active Standard order’s model, version, and region; model changes stay within the publisher, and approval, available capacity, processing time, and restrictions near expiry matter. A change request is not immediate failover capacity. Coordinate the term and assignment with retirement and upgrade plans under model version pinning. Self-hosting instead requires decisions about accelerator capacity and operating costs, covered in capacity planning. Every Team Had Its Own Key places these choices within a shared platform’s quotas and cost allocation.

References:

• Google Cloud, Provisioned Throughput overview (Creative Commons Attribution 4.0).
• Google Cloud, Standard pay-as-you-go (Creative Commons Attribution 4.0).
• Google Cloud, Error code 429 (Creative Commons Attribution 4.0).
• Google Cloud, Provisioned Throughput: Supported models.
• Google Cloud, Purchase Provisioned Throughput.
• Google AI for Developers, Gemini API: Rate limits.
• AWS Builders Library, Timeouts, retries, and backoff with jitter.