Phase 9: Agent Evaluation & Observability · 60 min · Python · Anthropic SDK · Pydantic
LLM-as-Judge
Scale evaluation to thousands of outputs — but only if you calibrate the judge first.
Hiring signal: LLM-as-judge is the industry standard for scalable agent evaluation at companies like Anthropic, Google, and every serious AI engineering team. Understanding judge biases and calibration is what separates engineers who actually measure quality from those who ship and hope.
What you will learn
- Implement a reliable LLM-as-judge with a structured rubric and Pydantic-validated output
- Measure and mitigate judge biases: position bias, length bias, and self-bias
- Calibrate a judge against human annotations and compute 95% confidence intervals over batch results
The Problem
You have 500 agent outputs to evaluate after a prompt change. Human evaluation would take a week and cost thousands of dollars. You can't reduce them to exact-match metrics — the outputs are paragraphs of research synthesis, and you're evaluating dimensions like relevance, accuracy, and completeness. You need something that scales.
LLM-as-judge is the industry solution. You send each (task, output) pair to a more capable model and ask it to score the output against a rubric. At 500 evaluations, this takes minutes and costs a few dollars. At 50,000 evaluations — checking production quality every day — it's still viable. Human evaluation at that scale is not.
The catch is that LLM judges have systematic biases. They favor the first option in a comparison. They favor longer responses. They may favor outputs from their own model family. An uncalibrated judge produces confident-sounding scores that diverge significantly from human judgment. Engineering a reliable judge means designing the rubric carefully, testing for biases, and validating the judge against a held-out set of human annotations before you trust its aggregate scores.
The field is grounded in published research. G-Eval (Wang et al., 2023) established that GPT-4-based judges correlate well with human judgment on NLG tasks. MT-Bench and Chatbot Arena use LLM judges as a core evaluation mechanism. Understanding the architecture and failure modes of LLM judges is now an expected competency for senior AI engineers.
When to Use LLM-as-Judge
LLM-as-judge works best for evaluating complex, open-ended outputs where exact-match is impossible — summaries, research reports, multi-step reasoning. It is not suitable for factual verification against a known database (use exact-match or retrieval), nor for evaluating safety-critical outputs without human review. Always calibrate before trusting aggregate scores.
Judge Architecture
The judge receives a structured prompt containing: the original task, the agent's output, and an explicit evaluation rubric. It returns a structured JSON object with per-criterion scores, reasoning, and a confidence estimate.
class CriterionScore(BaseModel):
criterion: str
score: int = Field(ge=1, le=5)
reasoning: str
class JudgementOutput(BaseModel):
overall_score: float = Field(ge=1.0, le=5.0)
criteria_scores: list[CriterionScore]
reasoning: str
confidence: float = Field(ge=0.0, le=1.0)
Three design decisions matter here. First, use a more capable model as the judge than the one being evaluated. A judge that is less capable than the evaluated model will miss nuanced errors. If you're evaluating Haiku outputs, use Sonnet or Opus as the judge. Second, provide explicit evaluation criteria — don't ask the judge to "rate quality." A rubric like "Accuracy: are all factual claims correct and verifiable? 5 = all facts correct, 1 = significant errors" produces far more consistent scores than open-ended quality ratings. Third, the reasoning field is not decorative. Read the reasoning on a sample of evaluations: if the reasoning doesn't match the score, the judge is confused. The reasoning is your debug window.
A five-criterion rubric for research agent outputs: Relevance, Accuracy, Completeness, Conciseness, Citation quality. Each scored 1–5. Add this instruction explicitly: "Do NOT favor responses for being longer. A concise, accurate answer is better than a verbose one." Without this, length bias will inflate scores for padded responses.
Unlock the full lesson
You've read the first 2 sections. The rest of this lesson covers Judge Bias and Calibration, Batch Evaluation at Scale, Build It, What to Practice — plus a hands-on lab, quiz, and project artifact.
Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.
Browse all courses · View pricing · DeVenture Academy