Phase 8: Production AI Security Architecture · 60 min · dataclasses · enum · argparse
The Problem
Every prior lesson in this phase built a control: an architecture validator, a monitoring signal, a CI/CD gate, a supply chain scanner. Controls fail anyway. Guardrails get bypassed by a novel jailbreak. A retrieval index gets poisoned before anyone notices the output drift. An API key gets used for a slow, patient extraction campaign that looks like nothing until someone finally cross-references the query log. When one of these happens, the question stops being "how do we prevent this" and becomes "what do we do in the next sixty minutes, and who has to be told, and by when."
Most teams answer that question badly, for a specific reason: their incident response plan is either a generic SOC/AppSec runbook that has nothing to say about a poisoned retrieval index or a model that's leaking its system prompt, or it exists only as tribal knowledge in the head of whoever built the guardrail pipeline. Neither survives contact with an actual incident at 2am when that person is on a plane.
AI systems also carry regulatory obligations that a standard security incident often doesn't. The EU AI Act's Article 73 requires providers of high-risk AI systems to report serious incidents to a market surveillance authority — with materially shorter deadlines than most companies' existing breach-notification process assumes. If personal data was exposed, GDPR Art. 33/34's 72-hour clock starts running the moment you become aware, regardless of whether you have a report ready. NIST's AI RMF expects every confirmed incident to feed back into the Govern function's risk register, not just get fixed and forgotten. A playbook that doesn't compute these obligations from the specific facts of the incident either over-reports (burning legal and PR resources on incidents that don't meet the bar) or under-reports (missing a deadline that has real financial and legal consequences).
This lesson builds a playbook generator for the three incident categories unique to AI systems — prompt injection, data poisoning, and model extraction — following the same detection → triage → containment → eradication → recovery → post-mortem structure a mature security team already uses, but with every step grounded in the specific artifacts this phase already built, and a regulatory-obligations calculator that reasons from facts instead of applying a blanket checklist.
Prompt injection incident response
A prompt injection incident starts differently depending on whether it's direct (an attacker-authored prompt) or indirect (a payload smuggled in through a retrieved document, email, or tool output — the XPIA pattern from earlier in the course). The detection signal is usually SecurityMonitor.injection_rate_alerts from Lesson 2 crossing its rolling-window threshold, or a single request flagged critical by the input-layer classifier from Lesson 1.
Triage has one question that determines almost everything downstream: did the injection reach a tool call with side effects? An injection that only changes conversational tone is a high-severity annoyance. An injection that reaches send_email or delete_record through the guardrails-layer tool-authorization gate is critical, and containment means revoking the offending API key or session immediately — before root cause is even understood. Waiting to fully understand an attack before containing it is the single most common mistake in early-stage incident response, for AI incidents as much as any other kind.
Eradication isn't complete when the immediate bypass is patched. The observed payload has to become a permanent regression case in the CI/CD adversarial suite from Lesson 3, or the exact same technique works again the next time someone "temporarily" relaxes a filter to fix a false positive.
Data poisoning investigation
Poisoning incidents are detected differently from injection — usually a canary-prompt regression suite (a fixed set of prompts with known-good outputs, re-run on every retrain or index rebuild) shows an unexpected output shift, or model-layer drift monitoring flags an anomalous embedding or output distribution against a labeled baseline. The investigation's first job is locating the entry point: was this a direct contribution to a training set, a RAG document poisoning attack (Phase 4 material), or a compromised feedback loop that incorporates user signals into fine-tuning?
That distinction changes the entire remediation cost. A poisoned retrieval index can be fixed by quarantining the source document and re-embedding — cheap, fast, no retrain required. A poisoned fine-tune means rolling back to the last known-good checkpoint via the model-layer version pinning from Lesson 1, and that checkpoint itself needs re-verification against its recorded hash (Lesson 4) before it's trusted again — the checkpoint store, not just the training data, could be the compromised link.
Containment cost is the reason poisoning triage asks "where" before "how bad"
The same poisoning incident can cost an afternoon or a week depending entirely on which layer was poisoned. Locating the entry point — training set, fine-tune, or retrieval index — before deciding on a containment strategy isn't just methodical, it's the difference between a low-cost index rebuild and an unnecessary full retrain that burns a week nobody had budgeted for.
A canary-prompt regression suite flags an output-distribution shift after a retrieval index rebuild. Before choosing a containment strategy, the on-call engineer's first triage step is to determine whether the poisoned data entered through the training set, a fine-tuning feedback loop, or the retrieval index. Why does this determination have to come before containment is chosen, rather than after?
The lesson states this directly: "the same poisoning incident can cost an afternoon or a week depending entirely on which layer was poisoned." If the engineer contains first and investigates the entry point later, they risk either over-reacting (triggering a costly full retrain when quarantining one document and re-embedding would have fully solved a retrieval-index poisoning) or under-reacting (declaring the incident resolved after an index rebuild when the actual poison is baked into a fine-tuned checkpoint and will resurface). Sequencing "where" before "how bad" is what lets the response match the actual remediation cost instead of guessing.
Model extraction detection and response
Extraction campaigns rarely look like a single attack — they look like an unusual traffic pattern sustained over hours. The detection signal is a query-pattern anomaly (an API key issuing a large volume of near-duplicate or systematically-varied queries, consistent with the methodology Carlini et al. describe for extracting production language model internals) or a token-cost anomaly from SecurityMonitor with no legitimate session structure behind it.
A monitoring system flags an API key issuing 40,000 systematically-varied, low-diversity-session queries over six hours. The on-call engineer's first instinct is to spend an hour fully characterizing what the attacker extracted before taking any action, so the eventual incident report is complete and accurate. Is this the right sequencing?
Explanation: Containment and investigation are not sequential steps in a live incident -- they run in parallel, and containment of an active, ongoing campaign takes priority because every additional hour extends the attacker's window. This mirrors the prompt injection guidance earlier in this lesson: revoke the credential first, understand the full scope second. A security team that insists on complete understanding before acting will consistently contain incidents later than one that acts on the cheap, reversible step (rate-limiting a key) immediately and refines its understanding afterward.
Eradication for extraction is different from injection or poisoning in one important way: the fix usually isn't patchable after the fact. You can't un-expose logits that already left the API. The realistic eradication path is encoding the observed query-pattern signature as a permanent detection rule and evaluating whether the API's output verbosity (raw logits, full probability distributions) should be reduced going forward — a structural change to what the API exposes, not a one-time patch.
Post-mortem templates and regulatory reporting
Every incident type in this lesson ends the same way: a blameless root-cause analysis that identifies which layer of the architecture failed to catch the incident — a system-design finding, never a personnel finding — followed by a regulatory-obligations assessment computed from the specific facts of the incident.
That computation matters more than it sounds. A prompt injection that exposed one customer's email through an unauthorized send_email call triggers GDPR's 72-hour breach notification clock (personal data exposed) and, if the system qualifies as high-risk under the EU AI Act, a 15-day serious-incident report to the relevant market surveillance authority. A data poisoning incident that only shifted comparison answers about product safety ratings, with no personal data involved and no high-risk classification, triggers neither — but every incident, regardless of external reporting thresholds, updates the AI system's risk register under NIST AI RMF's Manage function. Getting this right — not over-reporting incidents that don't meet the bar, and never under-reporting ones that do — is exactly the kind of judgment an AI Security Engineer is expected to exercise under time pressure, and exactly what this lesson's _assess_regulatory_obligations method encodes as reusable logic instead of expecting a human to recall four different frameworks' trigger conditions from memory during an active incident.
Build
Build it
The code artifact below is an IncidentPlaybookGenerator that takes a short IncidentContext (what was detected, when, how, how severe) and produces a full structured IncidentReport — detection, triage, containment, eradication, recovery, blameless post-mortem, and regulatory reporting — for each of the three AI-specific incident categories. Every step is grounded in an artifact from earlier in this phase, and the regulatory obligations are computed from the incident's actual facts rather than a fixed checklist.
"""
lesson_08_05_incident_playbook.py
An IncidentPlaybookGenerator that turns a short incident context (what was
detected, when, how severe) into a full structured AI security incident
response document -- detection, triage, containment, eradication, recovery,
blameless post-mortem, and regulatory reporting obligations -- for three
incident categories unique to AI systems: prompt injection, data poisoning,
and model extraction.
This is not a static template. Each incident type has its own
detection/triage/containment/eradication/recovery playbook grounded in the
artifacts built earlier in this phase (the SecurityArchitecture validator
from Lesson 1, the SecurityMonitor signals from Lesson 2, the CI/CD gates
from Lesson 3, and the supply chain scanner from Lesson 4), and the
regulatory-obligations section is computed from the specific facts of the
incident (was personal data exposed, is this a high-risk AI system) rather
than a fixed disclosure checklist.
Run:
python lesson_08_05_incident_playbook.py # demo: one report per incident type
python lesson_08_05_incident_playbook.py --type prompt_injection \\
--incident-id INC-2026-014 --severity critical \\
--summary "Indirect injection via a retrieved support ticket exfiltrated a customer email thread."
No external services, no API keys, no network calls.
"""
from __future__ import annotations
import argparse
from dataclasses import dataclass, field
from datetime import datetime, timezone
from enum import Enum
# ---------------------------------------------------------------------------
# Incident model
# ---------------------------------------------------------------------------
class IncidentType(str, Enum):
PROMPT_INJECTION = "prompt_injection"
DATA_POISONING = "data_poisoning"
MODEL_EXTRACTION = "model_extraction"
@dataclass
class IncidentContext:
incident_id: str
incident_type: IncidentType
detected_at: str
detected_by: str
severity: str # "critical" | "high" | "medium"
affected_systems: list[str]
summary: str
evidence: list[str] = field(default_factory=list)
personal_data_exposed: bool = False
high_risk_ai_system: bool = True
@dataclass(frozen=True)
class RegulatoryObligation:
framework: str
trigger: str
deadline: str
action: str
@dataclass
class IncidentReport:
context: IncidentContext
detection_steps: list[str]
triage_steps: list[str]
containment_steps: list[str]
eradication_steps: list[str]
recovery_steps: list[str]
root_cause_analysis: str
corrective_actions: list[str]
regulatory_obligations: list[RegulatoryObligation]
def to_markdown(self) -> str:
c = self.context
lines = [f"# Incident Response Report — {c.incident_id}", ""]
lines.append(f"**Type:** {c.incident_type.value} · **Severity:** {c.severity.upper()}")
lines.append(f"**Detected:** {c.detected_at} by {c.detected_by}")
lines.append(f"**Affected systems:** {', '.join(c.affected_systems)}")
lines.append("")
lines.append("## Summary")
lines.append(c.summary)
if c.evidence:
lines.append("")
lines.append("**Supporting evidence:**")
for e in c.evidence:
lines.append(f"- {e}")
lines.append("")
lines.append("## 1. Detection")
for step in self.detection_steps:
lines.append(f"- {step}")
lines.append("")
lines.append("## 2. Triage & Scoping")
for step in self.triage_steps:
lines.append(f"- {step}")
lines.append("")
lines.append("## 3. Containment")
for step in self.containment_steps:
lines.append(f"- {step}")
lines.append("")
lines.append("## 4. Eradication")
for step in self.eradication_steps:
lines.append(f"- {step}")
lines.append("")
lines.append("## 5. Recovery")
for step in self.recovery_steps:
lines.append(f"- {step}")
lines.append("")
lines.append("## 6. Post-Mortem")
lines.append("")
lines.append("**Root cause analysis (blameless):**")
lines.append(self.root_cause_analysis)
lines.append("")
lines.append("**Corrective actions:**")
for action in self.corrective_actions:
lines.append(f"- {action}")
lines.append("")
lines.append("## 7. Regulatory & Compliance Reporting")
if not self.regulatory_obligations:
lines.append("- No external regulatory reporting obligation identified for this incident's facts.")
else:
lines.append("| Framework | Trigger | Deadline | Required action |")
lines.append("|---|---|---|---|")
for ob in self.regulatory_obligations:
lines.append(f"| {ob.framework} | {ob.trigger} | {ob.deadline} | {ob.action} |")
lines.append("")
return "\n".join(lines)
# ---------------------------------------------------------------------------
# Type-specific playbooks
# ---------------------------------------------------------------------------
#
# Each playbook is grounded in an artifact from an earlier lesson in this
# phase so the response plan is executable, not aspirational: it names the
# specific detector, config, or gate an on-call engineer would actually run.
PLAYBOOKS: dict[IncidentType, dict[str, list[str]]] = {
IncidentType.PROMPT_INJECTION: {
"detection": [
"SecurityMonitor.injection_rate_alerts (Lesson 2) crosses the rolling-window threshold, or a single request is flagged critical by the input-layer injection classifier (Lesson 1, input layer).",
"A user or support ticket reports the assistant revealing system-prompt content, taking an unauthorized action, or responding off-policy.",
"Guardrails-layer PII or tool-authorization logs show a blocked-then-retried pattern consistent with an attacker iterating on a jailbreak.",
],
"triage": [
"Pull the flagged request(s) and every adjacent request from the same session/API key from the audit layer (Lesson 1) — full prompt, retrieved context, tool calls, and model version.",
"Classify direct injection (attacker-authored prompt) vs. indirect injection (payload embedded in a retrieved document, email, or tool output — the XPIA pattern from Phase 2/4).",
"Determine scope: single session, or a sustained campaign — cross-reference the injection_rate_alerts burst window against any correlated cost_anomalies (Lesson 2); a scripted attacker often drives both.",
"Assign severity: critical if the injection reached a tool call with side effects (send_email, delete_record, issue_refund) or exfiltrated data; high if it only altered conversational output.",
],
"containment": [
"If the injection reached a tool-call authorization gate, revoke or rate-limit the API key/session immediately — do not wait for root cause.",
"If the attack is a sustained campaign, temporarily tighten guardrails-layer topic restriction and tool authorization to a stricter allowlist, accepting a higher false-positive rate until eradication is confirmed.",
"Preserve the full session transcript, retrieved context, and tool-call log before any rollback — this is the evidence base for both root cause and any regulatory notice.",
],
"eradication": [
"Patch the specific bypass and add the observed payload as a new AdversarialCase in the CI/CD adversarial suite (Lesson 3) so it is regression-tested forever, not just fixed once.",
"If this was indirect injection, fix the source-tagging gap (Lesson 1, input/guardrails layers) so untrusted retrieved content is structurally prevented from being treated as an instruction.",
"Re-run the full CI/CD regression gate (Lesson 3) against the patched config and confirm the new case, plus the entire existing suite, passes before redeploying.",
],
"recovery": [
"Re-enable normal traffic in stages, watching injection_rate_alerts (Lesson 2) at a lowered alert threshold for 24-48 hours.",
"Confirm the architecture score for the input and guardrails layers (Lesson 1) has improved and the specific critical gap that allowed this incident is closed.",
"Notify affected customers/users if the incident met the disclosure bar determined in the regulatory-reporting step below.",
],
"corrective_actions": [
"Add the payload to the jailbreak library and injection-detector regression suite (Phase 2) so this exact technique is caught pre-merge going forward.",
"Review whether the trust-boundary tag genuinely persisted through the model and guardrails layers, or only existed at ingestion — this is the most common root cause of injection incidents that 'shouldn't have worked'.",
"Schedule this exact scenario into the next red-team engagement (Phase 6 methodology) rather than treating the point fix as sufficient.",
],
},
IncidentType.DATA_POISONING: {
"detection": [
"A canary-prompt regression suite (fixed prompts with known-good outputs, run on every retrain or index rebuild) shows an unexpected output-distribution shift.",
"Model-layer drift monitoring (Lesson 1/2 pattern) flags an anomalous shift in embedding or output distribution against the labeled baseline.",
"A data engineer or annotator flags an unusual concentration of near-duplicate or coordinated-looking submissions in a user-feedback or RLHF-style training data source.",
],
"triage": [
"Identify the poisoned data's entry point: direct training-set contribution, RAG document ingestion (Phase 4 material), or a fine-tuning/RLHF feedback loop that incorporates user signals.",
"Determine blast radius: does the poisoned data affect the base model, a fine-tune, or only the retrieval index (far cheaper to remediate — delete and re-embed vs. retrain)?",
"Check whether the poisoning is targeted (a backdoor/trigger phrase, Phase 3 material) or broad (label flipping / distributional attack) — this changes both the eradication path and severity.",
"Assign severity: critical if the poisoned model or index is already serving production traffic and producing incorrect or manipulated outputs to end users.",
],
"containment": [
"If the poisoning is in the retrieval index: immediately quarantine the affected document source and remove it from the live index — a fast, low-cost containment step versus a full retrain.",
"If the poisoning is in a fine-tune or training set: roll back model-layer version pinning (Lesson 1) to the last known-good checkpoint immediately — do not wait for a clean retrain.",
"Freeze the ingestion pipeline for the affected data source (disable the RAG connector, pause the feedback-loop trainer) until the source is verified clean.",
],
"eradication": [
"Re-run model weight verification (Lesson 4) against the last known-good checkpoint's recorded hash before restoring it, in case the checkpoint store itself — not just the training data — was compromised.",
"Audit and clean the poisoned data source; if a specific contributor or document is identified as the injection vector, revoke its ingestion privileges.",
"Add a regression case to the canary-prompt suite and the CI/CD guardrail regression gate (Lesson 3) covering the observed poisoning behavior.",
],
"recovery": [
"Re-embed/rebuild the retrieval index (or redeploy the verified clean checkpoint) and re-run the full architecture review (Lesson 1) to confirm the model layer's drift_monitoring and version_pinning controls are green.",
"Monitor output-distribution drift at a tightened threshold for an extended period — poisoning effects can be subtle and reappear after a partial rollback.",
"Document exactly which downstream decisions or user-facing outputs may have been affected by the poisoned window, for both remediation and regulatory reporting.",
],
"corrective_actions": [
"Add provenance verification (signed commits, contributor allowlists, or submission-volume anomaly detection) to the ingestion pipeline for the affected data source.",
"Establish or tighten a canary-prompt regression suite that runs on every retrain or index rebuild, not only on major releases.",
"Review model weight verification (Lesson 4) coverage — confirm every checkpoint promotion path re-hashes and verifies before serving.",
],
},
IncidentType.MODEL_EXTRACTION: {
"detection": [
"A query-pattern signal flags a client issuing an anomalously large volume of near-duplicate or systematically-varied queries — consistent with the logit/embedding-extraction methodology described in Carlini et al., 'Stealing Part of a Production Language Model' (2024).",
"A token-cost anomaly (Lesson 2, cost_anomalies) from a single API key with no corresponding legitimate usage pattern (no session structure, uniform request timing).",
"Rate-limiting or WAF logs (Lesson 1, network layer) show systematic probing across the input space rather than organic user traffic.",
],
"triage": [
"Pull the full query log for the suspected API key/client and check for the signature of extraction methodology: queries engineered to cover embedding space, probe logit outputs, or systematically vary a single prompt dimension.",
"Estimate what was extractable: with API-only access, an attacker typically approximates output-layer parameters or distills a student model from input/output pairs, not raw weights.",
"Determine whether the queries also touched proprietary or customer-specific fine-tuned behavior, which raises both IP and data-exposure severity.",
"Assign severity: critical if the volume/pattern indicates a completed or near-complete extraction campaign rather than reconnaissance.",
],
"containment": [
"Revoke or aggressively rate-limit the offending API key immediately — the single highest-leverage containment step for an in-progress extraction campaign.",
"Tighten per-key request quotas and add response perturbation/rate-limiting at the network and auth layers (Lesson 1) for all keys until the campaign is confirmed stopped.",
"Preserve the full query log as evidence — extraction campaigns are usually distributed across time, and the log is the only way to later prove scope.",
],
"eradication": [
"Encode the identified query-pattern signature as a detection rule in the monitoring layer (Lesson 2) and, where feasible, as a CI/CD-tested rate-limit policy (Lesson 3).",
"Evaluate output-layer countermeasures (Lesson 1, output layer), such as reduced logit/probability exposure in the API response, which directly reduces extraction feasibility for future campaigns.",
"Rotate any credentials used across the extraction campaign and review the auth layer's key-scoping (Lesson 1) to ensure the offending key had no broader access than necessary.",
],
"recovery": [
"Confirm the new rate-limit and detection rules pass the CI/CD regression gate (Lesson 3) without materially degrading legitimate high-volume users (e.g., enterprise batch customers).",
"Monitor token-cost and query-pattern signals at a tightened threshold for an extended period — extraction campaigns are frequently retried from new keys or IPs.",
"Assess and document IP exposure with legal/product stakeholders — this determines the regulatory and contractual reporting path below.",
],
"corrective_actions": [
"Reduce API response verbosity (raw logits/probabilities) to the minimum required for legitimate use cases.",
"Add per-key behavioral rate-limiting keyed on query diversity/coverage, not just raw request count, since extraction campaigns are engineered to stay under simple rate limits.",
"Review terms-of-service audit rights around systematic querying, to support both future detection and any legal response.",
],
},
}
# ---------------------------------------------------------------------------
# Generator
# ---------------------------------------------------------------------------
class IncidentPlaybookGenerator:
"""Turns an IncidentContext into a full IncidentReport using the
type-specific playbook and facts-based regulatory assessment."""
def __init__(self, playbooks: dict[IncidentType, dict[str, list[str]]] | None = None) -> None:
self.playbooks = playbooks or PLAYBOOKS
def generate(self, context: IncidentContext) -> IncidentReport:
template = self.playbooks[context.incident_type]
return IncidentReport(
context=context,
detection_steps=template["detection"],
triage_steps=template["triage"],
containment_steps=template["containment"],
eradication_steps=template["eradication"],
recovery_steps=template["recovery"],
root_cause_analysis=self._root_cause_analysis(context),
corrective_actions=template["corrective_actions"],
regulatory_obligations=self._assess_regulatory_obligations(context),
)
def _root_cause_analysis(self, context: IncidentContext) -> str:
return (
f"{context.summary} This is a blameless analysis: the objective is identifying which "
f"layer(s) of the production architecture failed to catch a {context.incident_type.value} "
f"incident of {context.severity} severity affecting {', '.join(context.affected_systems)} — "
"not assigning individual blame. A control gap is a system design finding, not a personnel finding."
)
def _assess_regulatory_obligations(self, context: IncidentContext) -> list[RegulatoryObligation]:
"""Regulatory obligations are computed from the specific facts of the
incident, not applied as a blanket checklist -- this mirrors how a
real incident commander and legal/compliance partner would reason
through disclosure requirements."""
obligations: list[RegulatoryObligation] = []
if context.personal_data_exposed:
obligations.append(
RegulatoryObligation(
framework="GDPR Art. 33/34",
trigger="Personal data was exposed to an unauthorized party as part of this incident.",
deadline="Notify the supervisory authority within 72 hours of becoming aware (Art. 33); notify affected individuals without undue delay if the exposure is high-risk to their rights (Art. 34).",
action="File the breach notification with the relevant Data Protection Authority; draft the individual notice if the high-risk threshold is met.",
)
)
if context.high_risk_ai_system:
obligations.append(
RegulatoryObligation(
framework="EU AI Act Art. 73 (serious incident reporting)",
trigger="A serious incident involving a high-risk AI system placed on the EU market (e.g. significant harm to health, safety, or fundamental rights, or a systemic malfunction).",
deadline="Report to the relevant market surveillance authority within 15 days of becoming aware, with materially shorter windows for incidents involving death or serious disruption to critical infrastructure.",
action="File the serious incident report; preserve the full incident log and audit trail per the Art. 12 record-keeping obligation.",
)
)
obligations.append(
RegulatoryObligation(
framework="NIST AI RMF (Manage function)",
trigger="Any confirmed AI security incident, regardless of whether an external reporting threshold is met.",
deadline="Before the next scheduled risk review, or immediately for critical severity.",
action="Update the AI system's risk register and incident log; feed the root cause into the Govern function's next control review.",
)
)
if context.incident_type is IncidentType.MODEL_EXTRACTION and context.severity in ("critical", "high"):
obligations.append(
RegulatoryObligation(
framework="Contractual / customer notification",
trigger="Model extraction may expose proprietary model behavior, prompts, or customer-specific fine-tuning data to a competitor or malicious actor.",
deadline="Per the contractual security-incident SLA (commonly 24-72 hours).",
action="Notify affected enterprise customers per the data processing agreement; assess IP/trade-secret exposure with legal.",
)
)
return obligations
# ---------------------------------------------------------------------------
# Demo contexts
# ---------------------------------------------------------------------------
def _demo_contexts() -> list[IncidentContext]:
now = datetime.now(timezone.utc).isoformat()
return [
IncidentContext(
incident_id="INC-2026-041",
incident_type=IncidentType.PROMPT_INJECTION,
detected_at=now,
detected_by="SecurityMonitor.injection_rate_alerts (window rate 42%, threshold 15%)",
severity="critical",
affected_systems=["support-chatbot-api", "ticket-retrieval-index"],
summary=(
"An indirect prompt injection embedded in a customer support ticket instructed the "
"assistant to forward an internal escalation thread, containing another customer's "
"email and order history, to an external address via the send_email tool."
),
evidence=[
"injection_rate_alerts: window ending at request 2113 — 42% flagged (threshold 15%)",
"tool-call log: send_email invoked with a recipient outside the verified customer-contact allowlist",
],
personal_data_exposed=True,
high_risk_ai_system=True,
),
IncidentContext(
incident_id="INC-2026-052",
incident_type=IncidentType.DATA_POISONING,
detected_at=now,
detected_by="Canary-prompt regression suite (post-reindex run)",
severity="high",
affected_systems=["product-docs-rag-index"],
summary=(
"A batch of newly ingested vendor documentation contained a coordinated set of near-duplicate "
"pages designed to bias retrieval toward a competitor's product being described as unsafe, "
"shifting the assistant's comparison answers within two days of ingestion."
),
evidence=[
"canary prompt 'compare product X and Y safety ratings' output changed materially post-reindex",
"14 near-duplicate documents from a single low-reputation ingestion source flagged retroactively",
],
personal_data_exposed=False,
high_risk_ai_system=False,
),
IncidentContext(
incident_id="INC-2026-063",
incident_type=IncidentType.MODEL_EXTRACTION,
detected_at=now,
detected_by="Token-cost anomaly + query-diversity monitor",
severity="high",
affected_systems=["public-facing-completion-api"],
summary=(
"A single API key issued roughly 40,000 systematically-varied, low-diversity-session queries "
"over six hours, consistent with a logit-probing extraction campaign against the fine-tuned "
"customer-support model."
),
evidence=[
"cost_anomalies: sustained elevated request rate from key ak_7f21 with no session structure",
"query log: near-uniform prompt template with single-token perturbations across ~40k requests",
],
personal_data_exposed=False,
high_risk_ai_system=True,
),
]
# ---------------------------------------------------------------------------
# CLI / Demo
# ---------------------------------------------------------------------------
def _build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description="Generate an AI security incident response report.")
parser.add_argument("--type", choices=[t.value for t in IncidentType], default=None,
help="Generate a single report of this incident type. Omit to run the 3-incident demo.")
parser.add_argument("--incident-id", default="INC-0000")
parser.add_argument("--severity", default="high", choices=["critical", "high", "medium"])
parser.add_argument("--summary", default="")
parser.add_argument("--affected-systems", default="production-llm-app",
help="Comma-separated list of affected system names.")
parser.add_argument("--personal-data-exposed", action="store_true")
parser.add_argument("--high-risk-ai-system", action="store_true")
return parser
if __name__ == "__main__":
args = _build_parser().parse_args()
generator = IncidentPlaybookGenerator()
if args.type:
context = IncidentContext(
incident_id=args.incident_id,
incident_type=IncidentType(args.type),
detected_at=datetime.now(timezone.utc).isoformat(),
detected_by="manual CLI invocation",
severity=args.severity,
affected_systems=[s.strip() for s in args.affected_systems.split(",") if s.strip()],
summary=args.summary or f"A {args.type} incident was reported via the CLI with no summary provided.",
personal_data_exposed=args.personal_data_exposed,
high_risk_ai_system=args.high_risk_ai_system,
)
print(generator.generate(context).to_markdown())
else:
print("=" * 70)
print("DEMO: generating one full incident report per incident type")
print("=" * 70)
for ctx in _demo_contexts():
report = generator.generate(ctx)
print()
print(report.to_markdown())