Phase 8: Evaluation, Safety & Observability · 60 min · OpenAI/Anthropic API · promptfoo · pytest

Evaluating LLM Systems: Benchmarks & LLM-as-Judge

If you can't measure your LLM app, you can't improve it — you can only have opinions about it.

Real-world application: Builds rigorous evals for open-ended outputs instead of eyeballing demos

What you will learn

Explain why standard ML metrics fail for open-ended generation
Build a task-specific eval set that reflects real failure modes
Use reference-based and LLM-as-judge scoring appropriately
Validate a judge against human labels and control its biases
Turn evals into a regression gate in CI

The Problem

With a classifier you compute accuracy and you're done. With an LLM there's no single right answer — a summary can be good in infinitely many ways — so teams fall back on "it looks good to me," ship, and then can't tell whether the next prompt tweak helped or hurt. They change a prompt, the demo still looks fine, and a regression sneaks into production. Building real evals is the single highest-leverage skill in LLM engineering; practitioners like Hamel Husain argue evals — not prompts — are the actual moat. "How do you evaluate an LLM feature?" is now a defining interview question.

The Concept

Three ways to score generated output, from cheapest/narrowest to most flexible:

1. Reference-based metrics : compare to a gold answer (exact match, F1, BLEU/ROUGE)
   -> only works when there IS a canonical answer (extraction, classification, QA)
2. Programmatic / assertions: check verifiable properties (valid JSON, contains X,
   no PII, length, schema) -> cheap, deterministic, great for guardrails
3. LLM-as-judge            : a strong model scores outputs against a rubric
   -> handles open-ended quality (helpfulness, tone) where 1 and 2 can't

Use the cheapest method that captures what you care about; reserve the judge for genuinely subjective quality.

You're evaluating a RAG system that extracts structured data from invoices (vendor name, amount, date). Should you use reference-based metrics, programmatic assertions, or LLM-as-judge?

Data extraction has canonical answers — the invoice has a real vendor name, amount, and date. Reference-based metrics (exact match, F1) are cheapest and most accurate here. LLM-as-judge adds cost and variability for a task with clear right/wrong answers. Use assertions as a secondary check (valid JSON, correct types), but reference metrics should be primary.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Build, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card.