Phase 0: Security Fundamentals for AI · 60 min · Python · STRIDE · MITRE ATLAS
The Security Mindset Shift
Stop asking 'is this code correct.' Start asking 'can this model be manipulated.'
Hiring signal: The capstone threat model produced in this lesson is portfolio-ready — interviewers for AI Security Engineer and Red Team roles routinely ask candidates to walk through a document exactly like this, combining STRIDE, MITRE ATLAS, and attack-surface layering into one coherent assessment.
What you will learn
- Articulate the mindset shift from verifying correctness to assuming adversarial input by default
- Combine STRIDE, MITRE ATLAS, and the 6-layer attack surface model into a single coherent threat model document
- Produce a complete, capstone threat model for a toy customer support chatbot end to end
- Explain how this fundamentals toolkit hands off to the attack-specific phases that follow in this course
The Problem
Every engineer is trained to ask one question by default: does this work? Write the function, run the tests, ship it. That question is necessary but it is not sufficient for AI systems, and the gap between "correct" and "secure" is where every incident in this phase's earlier lessons happened.
The Chevrolet dealership chatbot worked. It responded fluently, stayed in character, and never crashed — by every normal engineering standard it was correct. It also agreed to sell a $76,000 truck for one dollar, because nobody had asked the second question: can a user make this bot say something the business never authorized? EchoLeak's Copilot integration also worked — it summarized documents and answered questions exactly as designed. The code had no bug in it. What it lacked was anyone asking: is this email content trustworthy enough to influence what the assistant does next?
This is the mindset shift the whole c3-00 phase has been building toward. A traditional software engineer's default posture is: inputs are usually well-formed, users are usually not adversaries, and correctness is the finish line. An AI security engineer's default posture inverts every one of those assumptions: inputs are adversarial until proven otherwise, "the user" might be a document, an email, or a poisoned knowledge-base article rather than a person at a keyboard, and correctness is the starting line — the real question is what a motivated attacker can make the correct code do.
You already have the vocabulary for this. STRIDE (lesson 1) gives you the threat categories. MITRE ATLAS (lesson 2) gives you the shared, precise technique names. The 6-layer attack surface model (lesson 3) gives you the map of where to look. Trust boundaries and blast radius (lesson 4) give you the discipline for deciding what to trust and how much damage a compromise can do. This lesson does not add a fifth framework — it asks you to run all four at once, on a single system, end to end, the way you would on the job.
Two Questions, Every Time
The mindset shift compresses into two questions you should ask about every component before it ships, not after an incident forces you to:
| Old default (AppSec correctness) | New default (AI security) |
|---|
| "Does this code handle the expected input correctly?" | "What is the worst input a motivated attacker could construct, and what does this code do with it?" |
| "Is this data source reliable?" | "Is this data source untrusted by default, regardless of who owns it — and if so, is it tagged as such everywhere it flows?" |
| "Did we test the happy path and the obvious edge cases?" | "Did we test what happens when the model itself is the thing being attacked — not just the code around it?" |
| "Who is authorized to call this function?" | "If the model decides to call this function based on attacker-influenced input, does authorization still hold?" |
Notice the pattern: every "new default" question assumes the attacker gets a turn. That is the entire mindset in one sentence — assume the attacker gets a turn, and design for what they do with it.
Correctness and security are orthogonal, not the same axis
A model can be perfectly correct — well-calibrated, low-hallucination, accurate on every benchmark — and still be trivially manipulable, because manipulability isn't a correctness defect. It's a property of how much the system trusts its inputs. This is why "our eval scores are great" and "our system is secure" are two completely different claims, and why an AI security review is a separate exercise from a model quality review, not a subset of it.
A code reviewer approves a pull request that adds a new tool to an agent, noting "the function itself is well-tested and handles malformed arguments gracefully, so this is safe to merge." Under the mindset shift described in this lesson, what's missing from that review?
This is the exact gap illustrated by "read_order -> summarize -> refund_order" from the capstone model: every individual function can be correct, well-tested, and robust to malformed input, while the composition is still exploitable because the decision to call it was made by a model that can be steered by attacker-controlled text. A code-correctness review checks the function; a security review has to also check what happens when the caller's judgment itself is the attack surface.
Unlock the full lesson
You've read the first 2 sections. The rest of this lesson covers Why This Toolkit, Not Just Vibes, The Capstone: One System, All Four Frameworks, Build It, What to Practice — plus a hands-on lab, quiz, and project artifact.
Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.
Browse all courses · View pricing · DeVenture Academy