Definition

An AWS SageMaker endpoint is a fully managed API endpoint where you can deploy a trained ML model and make real-time or batch predictions.

  • SageMaker handles infrastructure, scaling, and serving so you don’t need to manage servers.
  • You send input data → endpoint runs inference → returns predictions.

Types of Endpoints in SageMaker

  1. Real-Time Inference Endpoint
    • Always-on HTTPS endpoint.
    • Low-latency predictions (e.g., fraud detection, chatbots).
    • Autoscaling supported.
  2. Asynchronous Inference Endpoint
    • For large payloads or long-running inference jobs.
    • You upload data to S3 → endpoint processes it → stores results in S3.
    • Useful for video processing, large document analysis.
  3. Batch Transform (Offline Inference)
    • Run inference on a large dataset stored in S3.
    • No live endpoint; results saved back to S3.
    • Good for scheduled jobs (e.g., scoring all customers monthly).

How It Works

  1. Train or Import Model
    • Train in SageMaker or bring your own model (TensorFlow, PyTorch, Scikit-learn, XGBoost, ONNX).
    • Save model artifacts to S3.
  2. Create Model in SageMaker
    • Package model + inference script + dependencies.
    • Register as a “Model” object.
  3. Deploy to Endpoint
    • Create an endpoint configuration (instance type, scaling policy).
    • Deploy model → SageMaker provisions containers/instances.
  4. Invoke Endpoint
    • Use AWS SDK (boto3) or REST API.
import boto3
sm = boto3.client("sagemaker-runtime")

response = sm.invoke_endpoint(
    EndpointName="my-endpoint",
    ContentType="application/json",
    Body='{"data":[1.0, 2.0, 3.0]}'
)
print(response['Body'].read())

Benefits

  • Fully managed: no need to manage EC2 servers.
  • Scalable: auto-scaling handles traffic spikes.
  • Flexible: supports multiple frameworks and custom Docker images.
  • Secure: integrates with IAM, VPC, encryption.
  • Cost control: pay only for instance time (or use async/batch to save).

Example Use Cases

  • Real-time fraud detection (low latency).
  • Recommendation APIs (autoscaling for traffic spikes).
  • Medical image classification (asynchronous, large payloads).
  • Monthly churn scoring for millions of users (batch transform).

Summary
AWS SageMaker endpoints = managed APIs to deploy ML models for real-time, async, or batch inference.
They handle scaling, security, and infrastructure so you can focus on the model itself.