Phase 5: Security Testing AI-Generated Code · 50 min · Python · AI models (Claude, GPT-4o) · prompt engineering

The Generation-Review Asymmetry

AI generates vulnerabilities 55.8% of the time but detects them 78.7% of the time in review mode. Exploit the gap.

Hiring signal: Engineers who understand the generation-review asymmetry can exploit it for dual-mode security testing, catching 78.7% of vulnerabilities that AI introduces during generation.

What you will learn

Explain the generation-review asymmetry: 55.8% generation rate vs 78.7% review detection rate
Apply the dual-mode testing approach: generate code, then have the same model review it
Understand why 'write secure code' prompts only reduce vulnerabilities by 4 points (insufficient)
Quantify the gap between generation and review modes for security testing

The Problem

AI models have a fascinating behavioral asymmetry: when asked to generate code, they produce vulnerabilities 55.8% of the time. But when asked to review code (including their own), they detect vulnerabilities 78.7% of the time. This 22.9-point gap is the generation-review asymmetry, and it's the foundation of dual-mode security testing.

The Numbers

Mode	Vulnerability Rate	What it means
Generation	55.8%	AI generates vulnerable code more than half the time
Review	78.7% detection	AI detects vulnerabilities in code 78.7% of the time
Gap	22.9 points	AI is better at finding bugs than avoiding them

Why the Asymmetry Exists

In generation mode, the AI optimizes for code that looks correct and compiles. It pattern-matches from training data, picking the most common implementation -- which often has security flaws (CWE-1236, CWE-916).

In review mode, the AI is given specific code to analyze. It can focus on security properties, compare against known vulnerability patterns, and reason about edge cases. The review task activates different reasoning pathways than the generation task.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Dual-Mode Testing Approach, Build It — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.