Phase 9: LLMOps & Production Deployment · 70 min · Docker · GitHub Actions · FastAPI

Containerization & CI/CD for AI Systems

If it doesn't run in a container with one command, it isn't deployable.

Hiring signal: Production deployment readiness — Docker, CI/CD, reproducibility

The Problem

An engineer demos an AI feature from their laptop. It works perfectly. The team tries to deploy it. It fails because of a missing Python package, a different CUDA version, environment variables that only exist locally, and a model file that was downloaded manually. The engineer says, "It works on my machine."

Containerization solves "works on my machine" permanently. CI/CD ensures every change is tested and deployable automatically.

The Concept

Why AI Containerization Is Harder Than Web Containerization

Containerizing a web app is straightforward: Python, your dependencies, and your code in a Docker image. Containerizing an AI system adds three complications:

GPU dependencies: if your model needs a GPU, the container needs CUDA drivers, cuDNN, and a matching PyTorch/TensorFlow build. Get one version wrong and the container crashes on startup. NVIDIA provides base images (nvidia/cuda:...) with matching CUDA versions, but you must ensure your ML framework version matches the CUDA version in the base image.

Model weights: a 7B parameter model is 14GB in FP16. You can either bake the weights into the image (large image, slow to build/push), download them at startup (adds 30-60s to cold starts), or mount them from a shared volume (fast but requires volume infrastructure). Each approach has tradeoffs.

Reproducibility: ML results should be reproducible — the same code, same model, same data should produce the same output. But if your Dockerfile uses pip install transformers without a version pin, a new release could change behavior silently. Lock all dependencies with exact versions (transformers==4.44.0, not transformers>=4.0).

The Deployment Stack

Your Dockerfile copies all code, then runs pip install -r requirements.txt. Every code change triggers a full reinstall of all dependencies (5 min rebuild). How do you fix this?

Docker caches layers in order. If you COPY all code before pip install, any code change invalidates the cache for the install layer. By copying only requirements.txt first, the pip install layer is cached and reused on every code-only change. The app code COPY goes last, so only that thin layer rebuilds. Rebuild drops from 5 min to ~10 sec.

Why Containers for AI?

Problem	Container Solution
Different Python versions	Exact Python version in image
Missing system libraries	All deps installed in Dockerfile
GPU driver mismatch	NVIDIA base images with matching CUDA
Environment variable drift	Explicit ENV in Dockerfile + secrets management
"Works on my machine"	Identical environment everywhere
Model file not found	Model downloaded or mounted at build/runtime

CI/CD for AI Systems — What's Different?

Traditional CI/CD runs test → build → deploy. AI systems add:

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Build It, Use It, Ship It, Evaluation, Exercises, Key Terms, Common Pitfalls, Interview Framing — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.