Phase 2: Static Analysis & Automated Gates · 55 min · Python · Semgrep · SonarQube

SAST for AI Code

Six tools, 7.6% coverage. The gap is the lesson.

Hiring signal: Engineers who understand why 6 industry SAST tools combined catch only 7.6% of Z3-proven vulnerabilities in AI code demonstrate awareness that no single tool is sufficient and that multi-layer defense is mandatory.

What you will learn

Configure Semgrep with custom rules targeting AI-specific defect patterns
Run multiple SAST tools and compose their findings
Explain why 6 industry tools combined catch only 7.6% of Z3-proven vulnerabilities
Build a multi-SAST pipeline that maximizes coverage through tool diversity

The Problem

The Z3 formal verification study tested 3,500 AI-generated code artifacts across 7 models and found a 55.8% vulnerability rate with 1,055 formally proven exploitable vulnerabilities. Then they ran 6 industry SAST tools against the same code. Combined, all 6 tools caught only 7.6% of the Z3-proven vulnerabilities.

This means: 92.4% of confirmed vulnerabilities in AI-generated code pass through industry-standard SAST tools undetected. If your security strategy is "run Semgrep in CI," you are catching fewer than 1 in 10 vulnerabilities in AI code.

This is not because SAST tools are bad. It's because AI code has different defect patterns than human code, and SAST tools were designed for human code patterns. The lesson is not "don't use SAST" -- it's "use SAST as one layer, not your only layer."

SAST Tool Landscape for AI Code

Tool	Strengths	AI code limitations	Cost
Semgrep	Custom rules, fast, open-source	Pattern-based, misses semantic bugs	Free / $500/dev/yr Teams
SonarQube	Quality gates, technical debt tracking	Rules designed for human patterns	Free Community / $25k+/yr
Snyk Code	AI-powered, fast, good UX	Trained on human code, AI blind spots	$52/dev/yr
CodeQL	Semantic analysis, query language	Complex setup, slow on large repos	Free for OSS / GitHub Advanced
DevSkim	Microsoft, security rules	Limited language coverage	Free
Pysa (Meta)	Taint analysis, data flow	Python only, complex config	Free

The 7.6% detection gap

The study found that each tool individually catches 2-4% of Z3-proven vulnerabilities. Combined, they catch 7.6%. This is because:

Overlap: Tools detect the same easy-to-find vulnerabilities (low-hanging fruit)
Blind spots: Each tool misses entire categories (e.g., Semgrep misses semantic bugs, CodeQL misses pattern-based issues)
AI-specific patterns: SAST rules are written for human code patterns, not AI defect patterns

The 7.6% gap is the justification for multi-layer defense
If SAST caught 90% of vulnerabilities, you could stop there. But at 7.6%, SAST is necessary but not sufficient. You need SAST + property-based testing (Phase 3) + mutation testing (Phase 4) + formal verification (Phase 5) to close the gap. Each layer catches what the others miss. The 92.4% gap is the entire reason this course exists.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Semgrep: Custom Rules for AI Defect Patterns, Composing Multiple SAST Tools, Build It — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.