Phase 8: Production AI Security Architecture · 65 min · OWASP GenAI Security Project · NeMo Guardrails · OAuth 2.0

The Multi-Layer AI Security Architecture

Defense in depth isn't a diagram — it's seven layers that each assume the other six will fail.

Hiring signal: AI Security Architect and Staff AI Security Engineer interviews (the $170k-$250k tier) almost always include a design-review exercise: 'draw and defend the security architecture for this LLM application.' Candidates who can only name individual controls (a guardrail here, a WAF rule there) lose to candidates who can present a layered model, name the failure mode of each layer, and explain what compensates for it. This lesson gives you the reference architecture and a validator that turns it from a diagram into something you can run against a real config.

What you will learn

Define the seven layers of a production AI security architecture — network, auth, input, guardrails, model, output, audit — and the specific failure mode each layer exists to catch
Map concrete controls (WAF rules, OAuth scopes, schema validation, topic rails, drift monitors, output filters, audit retention) to each layer for a real LLM application
Build and run a validator that scores a system config against all seven layers and reports the specific gaps, not just a pass/fail
Present and defend a layered architecture in a design review by explaining what happens when any single layer is bypassed

The Problem

Every phase of this course so far has taught you to attack or defend one piece of an AI system: injection detection, RAG isolation, agent tool authorization, adversarial robustness, guardrail pipelines, red team methodology, compliance mapping. In production, none of those live alone. They are layers in a single request path, and the security of the system is determined by how those layers compose — not by how good any single layer is.

This is where most teams get it wrong. A team ships a strong prompt-injection classifier at the input layer and treats the system as "secure." Then a tool-output poisoning attack (Phase 4) walks straight through, because nothing at the guardrails or output layer was checking tool results. Or a team locks down guardrails carefully, then a model version bump silently changes refusal behavior with no anomaly detection to catch it (Phase 3, model layer) — the CrowdStrike-style "we shipped a change with no rollback path" failure, just for a model instead of a kernel driver.

Real incidents follow this pattern. Indirect prompt injection vulnerabilities like the 2025 "EchoLeak" class of issues in Microsoft 365 Copilot were exploitable specifically because a single layer — content ingested from email/documents — was trusted as instruction-equivalent, with no independent layer downstream to catch exfiltration via crafted markdown links. A layered architecture doesn't prevent every attack. It ensures that when one layer is bypassed, at least one other layer still has a chance to catch it before it becomes an incident.

The reference architecture below is the one you'll be expected to draw, defend, and — in this lesson — actually validate against a config, in an AI Security Architect design review.

The Seven Layers

A request into a production LLM application passes through seven layers, in this order:

#	Layer	Purpose	Typical controls
1	Network	Stop volumetric/infra attacks before the app sees them	WAF, DDoS protection, API gateway, TLS
2	Auth	Establish who is calling and what they can do	OAuth 2.0, scoped API keys, mTLS, key rotation
3	Input	Validate and classify everything before it becomes a prompt	Schema validation, injection detection, trust-tier tagging
4	Guardrails	Enforce policy independent of the model's own judgment	Topic rails, PII filtering, tool authorization, safety classifiers
5	Model	Control which model runs and detect behavioral drift	Version pinning, anomaly detection, drift monitoring
6	Output	Treat generated content as untrusted until validated	Response filtering, fact verification, output encoding
7	Audit	Make every decision reconstructable after the fact	Full logging, retention, tamper-evident storage

Each layer has a specific failure mode it exists to catch — not a generic "more security is better" justification. If you can't state the failure mode, you can't tell whether the layer is actually doing anything:

Network fails when unbounded traffic reaches the model endpoint directly — enabling scraping, credential stuffing, and denial of service on token spend.
Auth fails when any caller with a plausible-looking token gets full access — no scoping means a leaked customer-support key can call admin operations.
Input fails when untrusted content (user input, retrieved documents, tool output) reaches the model without being flagged as untrusted — this is the root cause of nearly every prompt injection incident from Phase 2.
Guardrails fail when the model is the only thing enforcing policy — one successful jailbreak (Phase 2) removes every restriction at once.
Model fails when a version or prompt-template change ships with no eval gate and no rollback — silent regressions in refusal behavior or output quality go undetected until a user reports them.
Output fails when generated content is rendered, executed, or stored without validation — OWASP's insecure output handling category (LLM02) covers exactly this: XSS, SSRF, and SQLi caused by trusting model output.
Audit fails silently, until the moment you need it: an incident happens and there's no record of the prompt, the retrieved context, or which model version produced the bad output. You cannot write an incident report (Phase 5 of this course) without this layer.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Trust Boundaries and Blast Radius, Presenting the Architecture in a Design Review, Build It, What to Practice — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.

Key terms in this lesson

Guardrails