Phase 3: The Generate-Validate Loop · 45 min · Claude Code · Python

The Validate Phase: Beyond "It Works"

"The AI said it's done" is not validation. Neither is "it ran without crashing."

Hiring signal: Naming the specific red flags in a code review — not a vague 'looks fine to me' — is exactly what distinguishes an engineer who validates AI-generated code from one who's just reading it for vibes.

What you will learn

Explain why an agent's self-report that code 'works' is not validation
Distinguish functional checkpoints (does it run) from acceptance validation (does it meet the spec)
Identify the four AI-generated code red flags: hardcoded secrets, swallowed errors, magic numbers, sycophantic comments
Build a static scanner that catches these red flags automatically instead of relying on manual review alone

Introduction

The Validate Phase: Beyond "It Works"

An engineer generates a payment-processing function, runs it once against a happy-path example, watches it print the correct total, and asks the agent, "does this look right to you?" The agent says yes, cites the passing example, and the engineer merges it. Three weeks later a customer with a $0.00 order (a fully-covered free trial) crashes checkout, because the generated code divides by the order total to compute a loyalty-points rate, and nobody — not the engineer, not the agent asked to self-assess — ever checked what happens at the edge the spec's test cases were supposed to cover. "Does this look right to you?" asked of the same model that wrote the code is not validation. It's the model re-reading its own work with the same blind spots that produced it, dressed up as a second opinion.

Two questions, and neither is "did the AI say it's fine"

Real validation asks two separate, concrete questions, and both have answers you can check independently of anything the agent claims. Does it run? — a genuinely low bar, but a real one: does the code execute without crashing on the test cases the spec actually specifies, not just the one input someone happened to try. Does it meet the spec's acceptance criteria? — the higher bar, and the one that actually matters: for each acceptance criterion written during the specify phase, is there a check (a test, a manual walkthrough against a concrete input) confirming the generated code satisfies it. Neither question is answered by the agent asserting confidence. Both are answered by running something and observing what actually happens.

The validation test that would have caught the $0.00 bug
"Does it meet the spec's acceptance criteria" only works if the spec's test cases actually include the edge cases that matter — a $0.00 order, an empty coupon list, a stacked-to-over-100% discount. Validation isn't just running tests; it's running the right tests, the ones the specify phase should have named. An agent asked "does this handle edge cases" will often say yes without checking any specific one — the fix is asking about a named, concrete edge case, not the category in the abstract.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Reading AI-generated code: four red flags, Why generated code exhibits these patterns more, not less, Build It — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.

Key terms in this lesson

Agent