Phase 5: Modern LLM Engineering · 65 min · OpenAI API · Anthropic API · Ollama

The LLM Landscape & Model Selection

The best model is the one that meets your cost, latency, and quality bar — not the one with the highest benchmark.

Hiring signal: Cost-aware model selection and tradeoff analysis

The Problem

A junior engineer defaults to GPT-4 for every feature. The product works beautifully in demos. Then the CFO asks why the AI bill is $47,000/month for a feature that could run on a model 10x cheaper. Meanwhile, a competitor ships the same feature with Claude Haiku at 1/20th the cost and 3x the speed.

Model selection is not about picking "the best model." It is about picking the right model for each task given your constraints: quality threshold, latency budget, cost ceiling, data privacy requirements, and deployment context.

The Concept

The Four Tiers of LLMs

The LLM landscape in 2024-2025 has four tiers, organized by capability and cost:

Frontier models (GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro) are the most capable — they handle complex reasoning, multi-step planning, and nuanced understanding. They're also the most expensive ($2-15 per million tokens) and slowest (1-5 seconds per response). Use them for the hardest 5-10% of queries where quality is paramount and cost is secondary.

Workhorse models (GPT-4o-mini, Claude Haiku, Gemini Flash) are the sweet spot for most production workloads. They're 10-20x cheaper than frontier models, 3-5x faster, and handle the vast majority of tasks — classification, extraction, summarization, simple Q&A — with near-frontier quality. This is where 80-90% of your queries should go.

Open-weight models (Llama 3.1, Mistral, Qwen, DeepSeek) can be self-hosted, giving you data privacy, fixed costs (GPU rental vs per-token pricing), and full control over inference. The tradeoff: you manage infrastructure, security patches, and model updates. At high volume (>1M requests/day), self-hosting is often cheaper than API calls.

Small/edge models (Llama 3.2 3B, Phi-3, Gemma 2B) run on laptops, phones, or edge devices. They're fast, free, and private, but limited in capability — suitable for simple classification, basic extraction, or offline scenarios.

You're building a customer support chatbot that handles 10,000 queries/day. Most are simple FAQs; 5% need complex reasoning. What model strategy minimizes cost while maintaining quality?

95% of queries are simple FAQs that a workhorse model (GPT-4o-mini, Claude Haiku) handles well at ~10x lower cost. Route those to the cheap tier and escalate only the 5% that need complex reasoning to the frontier model. This cuts costs by ~8x while maintaining quality on hard cases.

Selection Framework

Every model choice should answer five questions:

Dimension	Question	Example
Quality	What accuracy/quality threshold must the task meet?	"Must correctly extract 95% of invoice fields"
Latency	What response time does the user experience require?	"First token under 200ms for chat"
Cost	What is the per-request and monthly budget?	"$0.002/request, $5k/month max"
Privacy	Can data leave the organization?	"PII must stay on-premises"
Scale	How many requests per second at peak?	"200 RPS during business hours"

The Model Selection Matrix

Task Complexity vs. Cost Sensitivity:

                    Low Cost Sensitivity    High Cost Sensitivity
                    ┌─────────────────────┬──────────────────────┐
  High Complexity   │ Frontier models     │ Open-weight large    │
  (reasoning,       │ GPT-4o, Sonnet 3.5  │ Llama 70B, DeepSeek │
   multi-step)      │                     │                      │
                    ├─────────────────────┼──────────────────────┤
  Low Complexity    │ Workhorse models    │ Small/edge models    │
  (classification,  │ GPT-4o-mini, Haiku  │ Phi-3, Gemma, local │
   extraction)      │                     │                      │
                    └─────────────────────┴──────────────────────┘

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Build It, Use It, Ship It, Evaluation, Common Pitfalls, Exercises, Key Terms, Interview Framing — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.

Key terms in this lesson

Latency