Cloud Deployment for ML (AWS-First) · 60 min · AWS SageMaker · AWS Lambda · API Gateway

Deploying Models on AWS

There are three good ways to serve a model on AWS. Knowing which, and why, is the skill.

Hiring signal: Can ship a model to a real cloud endpoint and defend the choice

What you will learn

Deploy a model as a SageMaker endpoint (real-time and serverless)
Serve lightweight inference with Lambda + API Gateway
Run a custom container on ECS/Fargate
Choose among the three by latency, cost, scale, and control
Package a model with its dependencies reproducibly

The Problem

"I trained a model" is a notebook. "I deployed a model" is a job. The leap is turning a model.pkl into an HTTP endpoint that's reliable, scalable, and not absurdly expensive. AWS gives you several ways to do this and the names blur together — so candidates either over-reach for SageMaker on a toy, or hand-roll a fragile EC2 box. The real skill is matching the serving pattern to the workload and being able to defend it.

This lesson covers the three patterns you'll actually use and how to choose.

The Concept

Choosing the Right Serving Pattern

The biggest mistake in model deployment is choosing the serving pattern based on what you've heard of, not what your workload needs. The three patterns — Lambda, ECS/Fargate, and SageMaker — are optimized for different workload shapes:

Lambda + API Gateway is best when traffic is spiky and the model is small. Lambda scales to zero — you pay nothing when there's no traffic. When a request arrives, AWS spins up a container, loads your model, and serves the request. This is perfect for a 50MB sklearn model that gets 0 requests at 3am and 200/sec at 2pm. The tradeoff: cold starts — the first request after idle takes 1-3 seconds while the container spins up. There are also package size limits (250MB) and runtime limits (15 min), so large models or long-running inference won't work.

ECS/Fargate is best when you need custom runtime control and steady traffic. You package your model in a Docker container and AWS runs it, handles scaling, and manages the infrastructure. You control the container — custom CUDA versions, system libraries, multi-model serving — but you don't manage SSH or OS patches. This is the good default for most production model serving: more control than Lambda, less management than EC2.

SageMaker endpoints are best when you want managed ML hosting with built-in features. SageMaker handles model loading, autoscaling, A/B testing, monitoring, and canary deployments out of the box. The tradeoff: it's the most expensive per hour because you pay for the managed service overhead, and real-time endpoints have idle costs — you're paying even when no requests come in. Use SageMaker when you want to focus on ML, not infrastructure.

You're deploying a sklearn model (50MB) that handles spiky traffic — 0 requests at night, 200/sec during peak hours. Cost is the priority. Which AWS deployment option is best?

Lambda scales to zero when there's no traffic (free at night) and automatically handles traffic spikes. The 50MB sklearn model fits within Lambda's package size limit. No idle cost = minimal bill for spiky workloads. EC2 and SageMaker charge for idle time. ECS/Fargate has some idle cost too. Lambda is the clear winner for small models with spiky traffic.

Pattern            Best when                         Tradeoff
-----------------  --------------------------------  -----------------------------
Lambda + API GW    spiky / light / small models      scales to zero; cold starts,
                   (sklearn, small embeddings)        size & runtime limits
ECS / Fargate      custom runtime, steady traffic,   full container control;
   (containers)     medium models                     you manage scaling config
SageMaker endpoint standard ML, want managed host,   least ops; cost of idle
                   autoscaling, A/B, monitoring        real-time endpoints

All three end at the same place: a request comes in, your model runs, JSON goes out. They differ in who manages the box, how it scales, and what idle costs.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Build It, Use It, Ship It, Evaluation, Exercises, Key Terms, Common Pitfalls, Interview Framing — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.