Vertex AI

Vertex AI is the Google Cloud name used for managed ML workflows covered by this entry. A custom-model path prepares data, runs a managed training job or imports an existing model, registers the artifact, deploys it to an endpoint and sends prediction requests. Batch prediction handles stored datasets; managed foundation-model APIs provide another path. As with SageMaker, the user controls data permissions, evaluation and workload configuration.

As of September 2026, Google’s former Vertex AI introduction URL redirects to Google Cloud’s Introduction to machine learning on Gemini Enterprise Agent Platform. The workflow below concerns the managed ML capabilities associated with the Vertex AI name; consult the current documentation for the naming and availability of a specific feature.

Separate training, model management, and serving

The Google Cloud ML overview distinguishes data preparation, managed training, and model management. AutoML uses managed datasets; custom training can use them but does not require them. The Model Registry tracks model versions and supports evaluation and deployment. A model version identifies an artifact and its associated configuration; it is not itself a running prediction service.

In a custom-training example, a team supplies a demand-forecasting application, historical data, and a compute configuration. A CustomJob runs that application in a prebuilt or custom container. The training code must save the resulting model to its intended artifact location. Hyperparameter tuning repeats training with different settings and a chosen objective; that objective still needs to represent the forecast quality the team wants.

Suppose the forecast must be ready before the daily inventory order. Evaluate on later periods than those used for training, then run predictions for the next ordering period and validate the result before the deadline. If an application instead needs predictions on demand, deploy the model to an endpoint. Deployment associates serving resources with the model. The endpoint is the address used by the application, while the deployed model represents the model on those resources. One endpoint can host multiple models, allowing a gradual traffic shift to a new version.

The service configuration does not determine whether an order quantity is sensible. The application still needs the same feature definitions, units, and preprocessing used in evaluation. Record the model version behind each prediction. If a new version increases errors or breaks the input contract, redirecting traffic to a tested prior version can restore the previous behavior, provided its serving resources and dependencies remain available. A model rollback cannot undo inventory orders already submitted.

Training and serving also need access to different resources. Give the workload identity access to the data and artifacts it actually uses, and give callers the required prediction permission. Diagnose permission failures, container failures, and prediction-quality failures separately. An endpoint can be healthy while forecasts deteriorate because the business or source data changed.

Estimate the work of migration

Every layer of an ML stack can be built against the platform SDK or against something portable. The same platform produces very different switching costs depending on which.

Consider the training code, data format, saved model, serving interface, workflow orchestration, identity, and monitoring separately. A standard model format and a container may preserve much of the numerical implementation, while job definitions, service accounts, routing, and alert configuration still depend on the target platform. Reusing a container does not prove that its accelerator libraries, storage access, or input/output behavior work elsewhere.

Assigning an assumed number of weeks to each layer and adding only platform-dependent layers can omit the cost of validating supposedly reusable components. A more useful estimate identifies the interface to replace, tests what can actually be reused, and includes validation and a period of parallel operation. Greater portability can reduce later changes while adding integration work now; the tradeoff depends on the parts the team expects to move.

Billing units and throughput assumptions

Foundation-model APIs and custom-model serving can use different billing units. Token billing is one common API pattern; dedicated serving capacity may be billed by time. Check the particular product, model, and configuration rather than assuming that every foundation-model API has the same pricing structure.

For an illustrative 400,000 daily calls, 900 input tokens and 180 output tokens per call, and assumed rates of 0.005 and 0.015 per thousand tokens, the daily charge is 400,000 × (900 × 0.005/1,000 + 180 × 0.015/1,000) = 2,880. Cutting input tokens by 40% lowers it to 2,160, a 25% reduction because output tokens remain. Capping output at 60 tokens also gives 2,160 under these assumptions, but changes what the application can say. These are invented rates, not a Google price quote.

For comparison, assume an instance sustains 0.8 requests per second at 4.60 per hour. Dividing the daily volume by 86,400 seconds gives 4.63 requests per second on average, requiring six instances and 662.40 per day in this simplified calculation. That is a lower-level capacity estimate, not proof of service equivalence: bursts, latency, model quality, and request lengths still need measurement.

Shorter prompts can reduce both token charges and compute work. Hourly billing falls only if paid capacity can be reduced or future capacity avoided. If batching really increases throughput fourfold, the same average-load calculation needs two instances and costs 220.80 per day; the gain must be measured, and waiting to form batches affects latency. If a correct application cache eliminates 30% of calls, this token example costs 2,016 and the hourly example needs five instances costing 552. Cache validity and any storage or lookup charges are additional conditions.

Commitments and utilization

A commitment can discount an eligible billing unit in exchange for reduced flexibility. The following calculation assumes fixed rates and fully interchangeable capacity, and is not a quote for a particular Google Cloud commitment.

Assume on-demand capacity costs 4.60 per used hour and a commitment costs 2.85 for each of 720 hours in a month, whether used or idle. The commitment costs 2,052. Equating that with 4.60 × used hours gives 446.1 hours, or about 62.0% utilization. At 300 used hours, on demand costs 1,380; at 600 hours, it costs 2,760. The same commitment lies between those amounts. This example assumes comparable capacity and no additional minimums or restrictions; actual eligibility and commitment terms may differ.

Under these assumptions, utilization below the break-even point makes the commitment more expensive. A forecast of high utilization is useful only if the workload can use the eligible capacity for the relevant period.

Low utilization can make committed capacity expensive per completed request, but high-scale cost also depends on model size, throughput, data movement and service prices. Compare platforms at measured utilization and equivalent service levels; accounting and cash flow are additional questions.

References

• Google Cloud, Serverless training overview
• Google Cloud, Deploy a model to an endpoint
• Google Cloud: Introduction to machine learning on Gemini Enterprise Agent Platform