AWS SageMaker

Amazon SageMaker AI manages infrastructure for building, training, and serving machine learning models. AWS renamed the original SageMaker service in December 2024; the broader Amazon SageMaker platform also includes data and analytics capabilities. This entry concerns the ML service. Existing sagemaker API names remain in use, as explained in AWS’s product overview.

From a training job to predictions

A training job runs learning code on selected compute resources. Its inputs include training data, algorithm or container, hyperparameters, and an execution role. Its output is a learned model artifact: files containing the parameters and other state needed later. SageMaker Training can run supported built-in algorithms, framework scripts, or a custom container. The managed job provisions the training environment; your code and configuration determine what is learned.

For example, a team predicting customer churn might prepare separate training and evaluation datasets, run a training job, and compare the resulting model with the current model on held-out customers. A successful job means the program completed, not that the new predictor is accurate, calibrated, or suitable for the business decision. Keep the data version, preprocessing, parameters, and evaluation result together so that a later deployment can be traced to the model actually assessed.

Deployment is a separate operation. A deployable model combines the saved artifact with inference code and its environment. The CreateModel API records this information and the role the service uses to access the image and artifacts. Creating that resource does not by itself start a live prediction server. A configured endpoint supplies the hosting resources; a batch transform job is another way to use the model without a persistent endpoint.

In the churn example, a nightly list of customers may be scored as a dataset. A website requesting a prediction during a session needs a request-response path instead. The inference mode follows that timing requirement. The application must send the feature representation expected by the deployed inference code and interpret the returned score using the intended decision policy. Training with one encoding and serving another can produce wrong predictions even when every cloud operation succeeds.

Managed infrastructure reduces provisioning work, but the team still decides which data may be used, who can invoke the model, how quality is assessed, and when to replace or retire it. Diagnose an unreadable artifact or incompatible container as a deployment problem; diagnose valid but worsening predictions using data and outcome evidence. Retraining is not a remedy for an incorrect input contract.

Training and serving have separate lifecycles

A finite training job and continuously provisioned inference accumulate costs differently. Neither must dominate every workload. The following assumed rates illustrate why the deployment schedule belongs in the estimate.

Suppose one training run uses 320 GPU-hours at an assumed 4.10 per hour, and runs at months 1, 4, 7, 10, 13, 16, 19, and 22. Eight runs cost 10,496. At an assumed 1.15 per instance-hour and 30-day months, one continuously running instance costs 1.15 × 24 × 30 × 24 = 19,872 over two years. Add assumed storage of 960 and pipeline costs of 2,160. Serving is then 59.3% of the total; three or ten instances raise that share to 81.4% or 93.6%, with the other costs fixed. These are chosen inputs, not AWS prices or measured traffic tiers.

With the illustrative instance prices and workload chosen here, serving accounts for 59.3% to 93.6% of the two-year total and overtakes cumulative training cost in month one or two. A large training run, occasional batch inference or a different deployment schedule can produce a different balance.

Use this calculation as a workload model: estimate training, serving and supporting services separately before deciding which platform features or costs dominate your project.

The managed premium is a number

This sensitivity calculation assumes a raw rate of 0.92 and a managed rate of 1.15 per instance-hour. Their 25% difference is an input, not a fixed SageMaker management fee or a current price quote.

For one instance over a 720-hour month, the assumed difference is (1.15 − 0.92) × 720 = 165.60. At an assumed engineering cost of 85 per hour, that equals about 1.9 hours. At 150 instances it becomes 24,840, or 292.2 hours. If self-managed operations additionally require a fixed 80 hours per month, their assumed labor cost is 6,800; the instance-price difference reaches that amount at about 41 instances. The fixed 80 hours are a scenario assumption, not staffing evidence.

Under those rates, the premium grows from the equivalent of 1.9 engineer-hours a month at one instance to 292 at 150. The break-even scenario additionally fixes self-managed operations at 80 hours per month. Real staffing effort, reliability obligations, support and utilization can change with fleet size.

Build-or-buy therefore needs measured labor and service costs as well as instance prices. This example illustrates a break-even calculation; it does not establish that self-hosting eventually wins for every team.

Compare configurations at equal service levels

Capacity settings also change the bill. The next comparison assumes predictable daily traffic, immediate scheduled scaling and identical per-instance throughput, excluding startup delays.

Assume one instance can sustain 28 requests per second, traffic is 35 requests per second for 21 hours and 220 for three hours, and the hourly rate is 1.15. Eight instances all day cost 220.80. Two all day cost 55.20 but provide only 56 requests per second at the peak. Scheduling two for 21 hours and eight for three hours costs (2 × 21 + 8 × 3) × 1.15 = 75.90. Using twelve during the peak costs 89.70. Both scheduled examples have sufficient nominal throughput; neither calculation establishes a latency target or accounts for scaling delay.

Eight always-on instances cost 220.80 a day. Scheduled scaling from two to eight costs 75.90 and two to twelve costs 89.70; both meet the assumed base and peak demand. Two instances without scaling meet base demand but only about 25% of peak demand. The cheaper two-instance case fails the peak demand, so its bill is not an equal-service comparison.

A useful platform evaluation follows the actual deployment workflow, measures throughput, and prices the capacity and operations needed to meet the service target. None of these examples establishes that configuration always matters more than vendor choice.

References

• AWS, What is Amazon SageMaker AI?
• AWS, Train a Model with Amazon SageMaker
• AWS, CreateModel
• AWS: Inference options in SageMaker AI