Phase 0: Security Fundamentals for AI · 55 min · Python · OWASP LLM Top 10

Attack Surface Mapping for LLM Applications & Agents

Every capability you add is a door. Map the doors before someone else does.

What you will learn

Decompose any LLM application into its 6 attack surface layers: model, prompt, retrieval, tool, agent, infrastructure
Map each OWASP LLM Top 10 category onto the layer(s) where it actually lives
Explain how attack surface grows non-linearly as capabilities are added to an agent
Prioritize which layers to defend first using exploitability and blast radius

The Problem

A support bot that only answers questions from a fixed FAQ has almost no attack surface — worst case, it says something embarrassing. Give that same bot a vector database of internal documents, a refund_order tool, and a planning loop that decides which tools to call, and you now have four qualitatively different places an attacker can push on: the model, the retrieved content, the tool arguments, and the agent's own decision-making. None of those existed as attack surface a week earlier when the bot only did FAQ lookup.

This is the pattern behind almost every serious AI security incident: teams reason about new capabilities in terms of what they enable, not what they expose. The EchoLeak vulnerability (CVE-2025-32711) existed because email content — a retrieval-adjacent input — was never modeled as something that could redirect tool use. The Chevrolet dealership chatbot incident existed because a thin prompt layer had no defense against being talked into a "legally binding" statement. Different layers, same root cause: nobody enumerated the layers before shipping.

The fix is a decomposition exercise you can run on any LLM application in about ten minutes, even one you've never seen before: break it into six layers, and ask what attacks live at each one.

The 6-layer model

Layer	What lives here	Example attacks
Model	Weights, fine-tuning, inference endpoint	Model theft via extraction queries, denial-of-service via adversarial inputs
Prompt	System prompt, user/assistant message channel	Direct prompt injection, jailbreaks, system prompt extraction
Retrieval	RAG corpus, vector DB, embeddings	Corpus poisoning, indirect injection via retrieved documents, embedding inversion
Tool	Function-calling schemas, arguments, connected systems	Insecure plugin design, tool argument injection, unsanitized tool output
Agent	Planning loops, memory, multi-agent orchestration	Excessive agency, goal hijacking, multi-agent injection propagation
Infrastructure	Hosting, network egress, secrets, CI/CD, supply chain	Credential leakage, poisoned dependencies, exfiltration via unrestricted egress

Each layer sits directly on top of the ones before it — a Tool-layer attack is only possible because the Prompt layer let attacker-influenced text reach a tool call, which is only possible because the Model layer will act on that text at all. Attack surface doesn't just add up across layers; it compounds.

Mapping OWASP LLM top 10 categories onto the layers

The OWASP LLM Top 10 (2023 v1.1) gives you ten named categories. The 6-layer model tells you where each one physically lives in a real system — which matters because "where" determines which team owns the fix.

Layer	OWASP Categories That Commonly Manifest Here
Model	LLM04 Model DoS, LLM10 Model Theft, LLM03 Training Data Poisoning
Prompt	LLM01 Prompt Injection, LLM06 Sensitive Information Disclosure, LLM07 Insecure Plugin Design
Retrieval	LLM01 Prompt Injection (indirect), LLM03 Training Data Poisoning (corpus poisoning), LLM06 Sensitive Information Disclosure
Tool	LLM07 Insecure Plugin Design, LLM02 Insecure Output Handling, LLM08 Excessive Agency
Agent	LLM08 Excessive Agency, LLM09 Overreliance, LLM01 Prompt Injection (multi-agent propagation)
Infrastructure	LLM05 Supply Chain Vulnerabilities, LLM04 Model DoS, LLM10 Model Theft

Notice LLM01 (Prompt Injection) appears at three different layers with three different mechanisms. That's not a mapping error — it's the single most important fact about prompt injection: it's not one vulnerability, it's a class of vulnerability that reappears at every layer where attacker-influenced text can reach a decision point.

Attack surface grows non-linearly, not linearly
Adding a single new capability rarely adds "one unit" of risk. A read-only FAQ bot has essentially one layer (Prompt) worth worrying about. Add retrieval and you've added a second layer, plus you've created new interactions between layers — retrieved content can now carry injected instructions that reach the Prompt layer's trust boundary. Add a refund tool and an agent loop, and you've added two more layers plus the interactions between all four. This is why "just add one more small feature" is a security-relevant sentence, not just a product one.

Diagramming a real system's attack surface

Walk the support bot from The Problem section through all six layers explicitly:

Model: a fine-tuned support model — could leak fine-tuning data or be extracted via repeated querying
Prompt: system prompt plus free-text user input — the user input is fully untrusted by definition
Retrieval: a knowledge-base vector store and a ticket-history retriever — both are editable by staff, and therefore poisonable by a compromised staff account or a malicious insider
Tool: refund_order (high blast radius, moves money) and lookup_order (read-only, low blast radius)
Agent: a planning loop deciding which tools to call, plus an escalation sub-agent for complex tickets — the hand-off to the sub-agent is itself a trust boundary
Infrastructure: the model provider's API credentials and the CI/CD pipeline that deploys prompt and tool changes — a supply-chain target if compromised

Eleven components, six layers, and already three separate trust boundaries worth diagramming (user → prompt, retrieval → prompt, agent → sub-agent). This is what "map the attack surface" produces in practice: not a vague sense of risk, but a specific list of components you can rank and defend in order.

A team adds a new capability to their support bot: it can now search the live web for competitor pricing information. Which layers does this new capability add or expand, and why does it matter that you name them specifically?

Live web search is a textbook case of a single feature expanding two layers at once. It's a Tool-layer addition (a new function the agent can call, with its own argument schema and failure modes), but it's also — and more dangerously — a Retrieval-layer addition, because the content that comes back from the web is now part of the model's context and is far less trustworthy than an internal, staff-curated knowledge base. Anyone can publish a web page designed to be retrieved and to contain injected instructions. Naming both layers matters because the fixes are different: the Tool layer needs argument/output validation, while the Retrieval layer needs the same untrusted-content tagging you'd apply to any indirect injection vector.

Prioritizing layers by exploitability × blast radius

You will never have budget to harden every layer equally in one sprint. Prioritize the same way you scored threats in the first lesson — by risk, not by which layer is most interesting to talk about:

Exploitability: how easy is it for an attacker to reach and influence this layer? Prompt is almost always highly exploitable — it's the layer every user touches directly.
Blast radius: what's the worst outcome if this layer is fully compromised? Tool and Agent layers tend to have the highest blast radius, because they're where "the model said something wrong" turns into "the model did something wrong" — an irreversible action in the real world.

A useful rule of thumb: Prompt is usually your highest-exploitability layer, but Tool and Agent are usually your highest-blast-radius layers. That combination — attacker-reachable content flowing into an irreversible action — is exactly the shape of the EchoLeak attack, and it's why "defend the Prompt-to-Tool path first" is close to universal advice across real AI security assessments.

Two layers in a system score identically on a naive "number of known vulnerabilities" count. The Prompt layer has high exploitability but the only consequence is an embarrassing off-topic response. The Tool layer has moderate exploitability but a compromised tool call can issue a real refund. Which should be prioritized first, and what scoring approach justifies that?

Raw vulnerability counts or component counts are a weak proxy for actual risk — they tell you where attention has historically gone, not where the real damage would occur if something went wrong today. Multiplying exploitability by blast radius, the same risk = likelihood x impact approach used for STRIDE threats in the first lesson of this phase, correctly surfaces the Tool layer here: even though it's somewhat harder to reach, the consequence of a successful attack (a real, possibly irreversible refund) vastly outweighs an off-topic response. This is the calculation that should drive sprint prioritization, not gut feeling about which layer 'feels' riskier.

Build

Build it

Before the full artifact, here's a minimal version of the same idea: score a handful of layers by exploitability x blast radius and rank them.

LAYER_RISK = {
    "Model": (2, 4),
    "Prompt": (5, 3),
    "Retrieval": (4, 4),
    "Tool": (3, 5),
    "Agent": (3, 5),
    "Infrastructure": (2, 5),
}

component_counts = {
    "Model": 1,
    "Prompt": 2,
    "Retrieval": 2,
    "Tool": 2,
    "Agent": 2,
    "Infrastructure": 2,
}

def priority_score(layer):
    exploitability, blast_radius = LAYER_RISK[layer]
    count = component_counts.get(layer, 1)
    return exploitability * blast_radius * max(1, count)

ranked = sorted(LAYER_RISK, key=priority_score, reverse=True)

print("Layer priority ranking (exploitability x blast_radius x components):")
for i, layer in enumerate(ranked, start=1):
    print(f"{i}. {layer}: score={priority_score(layer)}")

The code artifact lesson_00_03_attack_surface.py implements:

Layer enum for the 6 layers, with LAYER_DESCRIPTIONS and baseline LAYER_BASELINE_RISK (exploitability, blast_radius) priors
LAYER_OWASP_MAP — each layer mapped to the OWASP LLM Top 10 categories that most commonly manifest there
Component and AttackSurfaceMap dataclasses to decompose a real system
priority_score() and ranked_layers() — risk-ranks layers by exploitability × blast radius × component count
to_markdown() — renders a full attack surface report, ranked table first, layer detail second
A worked demo decomposing the RAG + agent support bot from this lesson into all 6 layers

"""
lesson_00_03_attack_surface.py

Classifies the components of an LLM application into the 6-layer AI attack
surface model (model, prompt, retrieval, tool, agent, infrastructure), maps
each layer to the OWASP LLM Top 10 (2023 v1.1 numbering: LLM01-LLM10)
categories that most commonly manifest there, and renders a prioritized
markdown attack-surface report.

Usage:
    python lesson_00_03_attack_surface.py

Requirements:
    none (standard library only)
"""

from __future__ import annotations

import argparse
from dataclasses import dataclass, field
from enum import Enum

class Layer(str, Enum):
    MODEL = "Model"
    PROMPT = "Prompt"
    RETRIEVAL = "Retrieval"
    TOOL = "Tool"
    AGENT = "Agent"
    INFRASTRUCTURE = "Infrastructure"

LAYER_DESCRIPTIONS = {
    Layer.MODEL: "The model itself — weights, fine-tuning, inference endpoint, training pipeline.",
    Layer.PROMPT: "The system prompt and the user/assistant message channel that reaches the model.",
    Layer.RETRIEVAL: "RAG corpus, vector DB, embeddings, and the retrieval pipeline feeding context to the model.",
    Layer.TOOL: "Function/tool calling: the schemas, arguments, and external systems tools connect to.",
    Layer.AGENT: "Multi-step planning, memory, and multi-agent orchestration built on top of the model.",
    Layer.INFRASTRUCTURE: "Hosting, network egress, secrets management, CI/CD, and the supply chain underneath everything else.",
}

# Baseline (exploitability, blast_radius) on a 1-5 scale, used only as a
# starting prior — a real assessment should adjust per system.
LAYER_BASELINE_RISK = {
    Layer.MODEL: (2, 4),
    Layer.PROMPT: (5, 3),
    Layer.RETRIEVAL: (4, 4),
    Layer.TOOL: (3, 5),
    Layer.AGENT: (3, 5),
    Layer.INFRASTRUCTURE: (2, 5),
}

# Which OWASP LLM Top 10 (2023 v1.1) categories most commonly manifest at
# each layer. Categories can and do appear at multiple layers — that overlap
# is itself a finding, not a bug in this mapping.
LAYER_OWASP_MAP = {
    Layer.MODEL: [
        ("LLM04", "Model Denial of Service"),
        ("LLM10", "Model Theft"),
        ("LLM03", "Training Data Poisoning"),
    ],
    Layer.PROMPT: [
        ("LLM01", "Prompt Injection"),
        ("LLM06", "Sensitive Information Disclosure"),
        ("LLM07", "Insecure Plugin Design"),
    ],
    Layer.RETRIEVAL: [
        ("LLM01", "Prompt Injection (indirect, via retrieved content)"),
        ("LLM03", "Training Data Poisoning (RAG corpus poisoning)"),
        ("LLM06", "Sensitive Information Disclosure"),
    ],
    Layer.TOOL: [
        ("LLM07", "Insecure Plugin Design"),
        ("LLM02", "Insecure Output Handling (unsanitized tool arguments)"),
        ("LLM08", "Excessive Agency"),
    ],
    Layer.AGENT: [
        ("LLM08", "Excessive Agency"),
        ("LLM09", "Overreliance"),
        ("LLM01", "Prompt Injection (multi-agent propagation)"),
    ],
    Layer.INFRASTRUCTURE: [
        ("LLM05", "Supply Chain Vulnerabilities"),
        ("LLM04", "Model Denial of Service (resource exhaustion at the infra level)"),
        ("LLM10", "Model Theft (via infrastructure access)"),
    ],
}

@dataclass
class Component:
    name: str
    layer: Layer
    description: str

@dataclass
class AttackSurfaceMap:
    system_name: str
    components: list[Component] = field(default_factory=list)

    def components_by_layer(self, layer: Layer) -> list[Component]:
        return [c for c in self.components if c.layer == layer]

    def present_layers(self) -> list[Layer]:
        seen = {c.layer for c in self.components}
        # keep canonical kill-chain-ish order
        return [l for l in Layer if l in seen]

    def priority_score(self, layer: Layer) -> int:
        exploitability, blast_radius = LAYER_BASELINE_RISK[layer]
        component_count = len(self.components_by_layer(layer))
        # more components at a layer widens the surface, so weight lightly by count
        return exploitability * blast_radius * max(1, component_count)

    def ranked_layers(self) -> list[Layer]:
        return sorted(self.present_layers(), key=self.priority_score, reverse=True)

    def to_markdown(self) -> str:
        lines = [f"# Attack Surface Map: {self.system_name}", ""]
        lines.append("## Layers Present, Ranked by Priority (exploitability x blast_radius x component_count)")
        lines.append("")
        lines.append("| Rank | Layer | Priority Score | Components | OWASP LLM Top 10 Categories |")
        lines.append("|---|---|---|---|---|")
        for i, layer in enumerate(self.ranked_layers(), start=1):
            comps = self.components_by_layer(layer)
            comp_names = ", ".join(c.name for c in comps)
            owasp = ", ".join(f"{cid} {cname}" for cid, cname in LAYER_OWASP_MAP[layer])
            lines.append(f"| {i} | {layer.value} | {self.priority_score(layer)} | {comp_names} | {owasp} |")

        lines += ["", "## Layer Detail", ""]
        for layer in self.ranked_layers():
            lines.append(f"### {layer.value}")
            lines.append(LAYER_DESCRIPTIONS[layer])
            lines.append("")
            lines.append("Components:")
            for c in self.components_by_layer(layer):
                lines.append(f"- **{c.name}**: {c.description}")
            lines.append("")
            lines.append("Relevant OWASP LLM Top 10 categories:")
            for cid, cname in LAYER_OWASP_MAP[layer]:
                lines.append(f"- {cid}: {cname}")
            lines.append("")

        return "\n".join(lines)

def build_demo_system() -> AttackSurfaceMap:
    """A RAG + agent customer support system, decomposed layer by layer."""
    return AttackSurfaceMap(
        system_name="RAG + Agent Customer Support Bot",
        components=[
            Component("Fine-tuned support model", Layer.MODEL, "Base model fine-tuned on past support tickets"),
            Component("System prompt", Layer.PROMPT, "Persona, constraints, and tool-use instructions"),
            Component("User chat input", Layer.PROMPT, "Free-text messages from customers, fully untrusted"),
            Component("Knowledge-base vector store", Layer.RETRIEVAL, "Embedded support articles retrieved per query"),
            Component("Ticket history retriever", Layer.RETRIEVAL, "Pulls a customer's past tickets into context"),
            Component("refund_order tool", Layer.TOOL, "Issues a refund given an order ID; highest blast radius tool"),
            Component("lookup_order tool", Layer.TOOL, "Read-only order status lookup"),
            Component("Planning loop", Layer.AGENT, "Decides which tools to call and in what order per turn"),
            Component("Escalation sub-agent", Layer.AGENT, "A second agent invoked for complex tickets"),
            Component("Model provider API", Layer.INFRASTRUCTURE, "External inference endpoint and its credentials"),
            Component("CI/CD pipeline", Layer.INFRASTRUCTURE, "Deploys prompt/tool changes; a supply-chain target"),
        ],
    )

def main() -> None:
    parser = argparse.ArgumentParser(description="Map an LLM application to the 6-layer attack surface model.")
    parser.add_argument("--output", default="attack_surface.md")
    args = parser.parse_args()

    asm = build_demo_system()
    markdown = asm.to_markdown()

    with open(args.output, "w") as f:
        f.write(markdown)

    print(f"Attack surface map written to {args.output}")
    print(f"Layers present: {[l.value for l in asm.present_layers()]}")
    print(f"Highest-priority layer: {asm.ranked_layers()[0].value}")

if __name__ == "__main__":
    main()

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Check Yourself — plus a hands-on lab, quiz, and project artifact.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card.