Phase 0: Platforms & Inference Economics · ~60 minutes · Python (stdlib · toy cascading router simulator)

Model Routing as a Cost-Reduction Primitive

A dynamic broker evaluates every request (task type, token length, embedding similarity, confidence) and sends simple queries to a cheap model, escalating complex ones to a frontier model.

Hiring signal: Can operate model routing as a cost-reduction primitive in production

What you will learn

Explain model cascading: cheap-first with confidence check, escalate on low confidence.
Enumerate the four routing signals (task classification, prompt length, embedding similarity to known-hard set, self-confidence from first-pass).
Compute expected blended cost at target routing split and quality loss tolerance.
Name the drift-monitoring metric (online quality gate) that catches cheap-model creep.

Introduction

Also called model cascading. Production case studies show 20-60% cost reduction at iso-quality across US/UK/EU deployments; a 30% routing efficiency improvement on high-volume SaaS turns into six-figure annual savings. The 2026 context is that LLM inference prices dropped ~10x per year — a GPT-4-class token went from $20/M to ~$0.40/M from late 2022 to 2026. Most of the drop is better serving stacks (Phase 17 · 04-09), not hardware. Routing is how you convert that price drop into margin without product regression. The failure mode is cheap-model drift: the route pushes 40% to a weaker model, quality drops 3-5% on reasoning tasks, no one notices for a quarter. Gate routes by online quality metrics, not just offline eval sets.

Type: Learn Languages: Python (stdlib, toy cascading router simulator) Prerequisites: Phase 17 · 01 (Managed LLM Platforms), Phase 17 · 19 (AI Gateways) Time: ~60 minutes

Objective

Learning objectives

Explain model cascading: cheap-first with confidence check, escalate on low confidence.
Enumerate the four routing signals (task classification, prompt length, embedding similarity to known-hard set, self-confidence from first-pass).
Compute expected blended cost at target routing split and quality loss tolerance.
Name the drift-monitoring metric (online quality gate) that catches cheap-model creep.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.