Phase 7: Responsible AI, Governance & Risk · 45 min · Incident response runbooks · NIST AI RMF · Python
The Problem
It's 9:14am. A screenshot is circulating on social media of your company's customer-support chatbot confidently telling a user that a prescription drug is safe to combine with alcohol — it isn't, and the interaction can be dangerous. The screenshot has 4,000 shares in ninety minutes and a health journalist has already DM'd your support account asking for comment. Your Slack has six people typing at once: an engineer wants to know if they should pull the model right now, legal wants to know if anyone said anything to the user yet, your VP wants a one-line summary for the CEO, and support is asking whether they should stop routing conversations to the AI at all.
Nobody in that thread is wrong to be asking their question. The problem is that if everyone acts on their own question independently, you get exactly the outcome that turns a bad hour into a bad month: an engineer silently patches a prompt without telling anyone, legal drafts a statement that contradicts what support already told the user, and the CEO finds out from Twitter instead of from you. The PM's job in an AI incident isn't to write the fix or the legal language — it's to be the one person holding the containment decision, the comms decision, and the fix timeline in the same head at the same time, because everyone else is only holding one piece.
The first hour: containment and fact-finding
The first hour has exactly one goal: stop the bleeding and get true facts, in that order. Not "figure out why," not "draft the public statement" — those come later and get worse if rushed.
- Contain first, explain later. If the failure is reproducible and ongoing (not a one-off), the PM's first real decision is whether to disable the feature, roll back to a previous model/prompt version, or add an emergency guardrail (a keyword filter, a stricter system prompt, human review before send) — a decision made with engineering, not for them. Containment doesn't require knowing root cause; it requires knowing the failure is real and ongoing.
- Get the facts nailed down before anyone talks publicly. How many users saw this. Is it reproducible or a one-off. Is it still happening right now or already stopped. A PM who tells support "we're not sure yet, don't confirm or deny details" for 30 minutes while facts get nailed down looks far better in six months than one who let someone guess in public and got the guess wrong.
- Notify in a specific order, not a group chat blast: engineering lead (can we contain it), legal/trust & safety (does this trigger disclosure obligations — see Lesson 3's regulatory triage, incident reporting can be a legal requirement not just a courtesy), then executive stakeholders (a factual one-paragraph summary, not a guess at the fix timeline), then a holding statement for anyone customer-facing (support, social) so they have something accurate to say rather than improvising.
"We're looking into it" is a real statement, not a stall tactic
The instinct under pressure is to say nothing until there's a complete answer. That's the wrong tradeoff during an active incident — silence for hours while a screenshot spreads reads as either not caring or not knowing, and both are worse than a short, honest holding statement: "We're aware of this report and are investigating. [Feature] has been temporarily [disabled/restricted] while we do." That sentence takes five minutes to get right and buys the hours you actually need to get the real answer right.
The first day: comms and stakeholder management
Once containment is real (not just decided — verified: the bad behavior has actually stopped for new users), the first day's job shifts to a public-facing statement that will be read by users, journalists, and possibly regulators, and an internal stakeholder brief that keeps leadership from freelancing their own version of events.
A good first-day statement does four things and nothing else: acknowledges the specific issue (not a vague "we take this seriously"), states what's been done so far (contained, disabled, rolled back — a verifiable fact), states what's being done next in terms the team can actually commit to (a specific investigation, not a specific fix date you haven't earned yet), and gives a real channel for affected users to get help. The single most common mistake here is promising a fix timeline before root cause is known — "will be fixed by Friday" said on day one, when the actual cause turns out to be a training-data issue that takes three weeks, converts one incident into two: the original failure, and then a broken promise on top of it.
Internally, the PM's job is translating between functions that are speaking different dialects under pressure: engineering wants to talk about root cause hypotheses (useful, but not what the CEO needs at hour 4), legal wants maximally hedged language (protects the company, but reads as evasive to a user if it's the only voice in the statement), and trust & safety may want to expand containment further than engineering thinks is necessary (that's a real tension to resolve explicitly, not paper over).
Six hours into an incident, engineering has rolled back the model and confirmed the specific failure has stopped occurring for new conversations. Legal wants to hold any public statement until root cause is fully understood, which could take days. What should the PM do?
Containment being verified is enough to say something true and useful publicly — a holding statement doesn't require root cause, it requires accurate facts about what happened and what's been done. Waiting days for legal's full sign-off on a complete explanation lets the story run uncontested in the meantime, which is a worse outcome for the company than a short, accurate, legal-reviewed holding statement now. Letting support freelance without a company-approved statement risks inconsistent or inaccurate public claims. Posting an unconfirmed root-cause theory is premature and risks having to walk it back, which damages credibility more than a "we're investigating" holding statement would.
The first week: root cause, fix, and closing the loop publicly
By the end of the first week, the incident has a real root cause, a real fix (not just a rollback), and — this is the step teams skip most often — a public follow-up that closes the loop with the same visibility the original failure got. If the initial screenshot had 4,000 shares and the resolution update gets posted quietly in a changelog nobody reads, the story that persists in public memory is "the company never explained what happened," even if internally it's fully resolved.
The PM owns the postmortem's translation into two different documents: an internal one (root cause, what monitoring would have caught this earlier — feeding back into the Measure function from Lesson 3's NIST AI RMF discussion — and what changed to prevent recurrence) and a public one (shorter, plain-language, focused on what changed for the user, not the internal engineering narrative). Skipping the public version because "the internal fix is done" is the same mistake as the model card mistake from Lesson 2: assuming the internal-audience document covers the external audience's actual question.
Three days into an incident, engineering has identified the root cause (a training-data gap) and estimates a real fix will take three more weeks. The team already published a day-one statement promising an update "within days." What should the PM do?
The specific danger the lesson warns about is a broken promise compounding the original incident — the fix here is not silence or a rushed bad fix, it's a prompt, honest correction: acknowledge the more complex root cause, give the real timeline, and describe what's protecting users in the meantime (the interim containment action). Silence lets the gap between the promise and reality get discovered by someone else, which reads worse than a proactive correction. Shipping an incomplete fix just to hit a self-imposed date recreates the core failure mode from Lesson 1: optimizing for a metric (schedule) instead of the actual outcome (a real fix).
Build
Build it
The code artifact lesson_07_04_incident_response.py implements an AI incident response tracker:
IncidentSeverity — severity levels tied to the consequence framework from Lesson 1IncidentTimeline — tracks containment status, key facts, and statements issued across the first hour/day/weekStakeholderNotification — who was notified, when, and with what informationrender_incident_log() — produces a markdown incident log suitable for the eventual postmortemreadiness_check() — a pre-incident checklist scoring whether a team actually has containment levers and an escalation path defined before they need them- A worked example: the prescription-drug-interaction chatbot scenario from the opening
"""
lesson_07_04_incident_response.py
An AI incident response tracker and pre-incident readiness checker.
Structures the first-hour/first-day/first-week PM workflow for a public
AI failure: containment status, stakeholder notifications, and public
statements, plus a readiness check a team can run before an incident
happens, not during one.
Reference: NIST AI RMF (https://www.nist.gov/itl/ai-risk-management-framework).
Usage:
python lesson_07_04_incident_response.py
No external dependencies, no network calls, no API keys required.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from enum import Enum
class IncidentSeverity(Enum):
LOW = "Low — isolated, no user harm, no public visibility"
MEDIUM = "Medium — limited user impact or visibility, contained quickly"
HIGH = "High — real user harm risk or significant public visibility"
CRITICAL = "Critical — active harm risk, high public visibility, possible regulatory exposure"
class ContainmentStatus(Enum):
NOT_STARTED = "Not started"
IN_PROGRESS = "In progress"
VERIFIED = "Verified — confirmed stopped for new users/conversations"
@dataclass
class StakeholderNotification:
stakeholder: str
notified_at: str # e.g. "T+15min"
what_they_were_told: str
@dataclass
class IncidentTimeline:
incident_name: str
description: str
severity: IncidentSeverity
containment_status: ContainmentStatus = ContainmentStatus.NOT_STARTED
containment_action: str = ""
notifications: list[StakeholderNotification] = field(default_factory=list)
holding_statement: str = ""
day_one_statement: str = ""
root_cause: str = ""
permanent_fix: str = ""
public_followup: str = ""
def add_notification(self, stakeholder: str, notified_at: str, told: str) -> None:
self.notifications.append(StakeholderNotification(stakeholder, notified_at, told))
def is_ready_for_public_statement(self) -> bool:
return self.containment_status == ContainmentStatus.VERIFIED
def readiness_check(
has_defined_containment_levers: bool,
has_defined_escalation_order: bool,
has_holding_statement_template: bool,
has_named_incident_owner: bool,
has_postmortem_process: bool,
) -> tuple[int, list[str]]:
"""Scores pre-incident readiness out of 5 and lists the gaps found."""
checks = {
"Defined containment levers (rollback, feature-flag off, guardrail patch) exist and are documented": has_defined_containment_levers,
"Escalation order (eng -> legal/T&S -> execs -> customer-facing teams) is defined in advance": has_defined_escalation_order,
"A holding-statement template exists so the first public words aren't drafted from scratch under pressure": has_holding_statement_template,
"A named incident owner (usually the PM) is defined, not assumed": has_named_incident_owner,
"A postmortem process exists that produces both an internal and a public-facing follow-up": has_postmortem_process,
}
score = sum(1 for v in checks.values() if v)
gaps = [k for k, v in checks.items() if not v]
return score, gaps
def render_incident_log(incident: IncidentTimeline) -> str:
lines = [
f"# Incident Log — {incident.incident_name}",
"",
incident.description,
"",
f"**Severity:** {incident.severity.value}",
f"**Containment status:** {incident.containment_status.value}",
f"**Containment action:** {incident.containment_action or 'Not yet specified'}",
"",
"## Stakeholder notifications",
"",
"| Stakeholder | Notified at | What they were told |",
"|---|---|---|",
]
for n in incident.notifications:
lines.append(f"| {n.stakeholder} | {n.notified_at} | {n.what_they_were_told} |")
lines += [
"",
"## Statements",
"",
f"**Holding statement (first hour):** {incident.holding_statement or 'Not yet issued'}",
"",
f"**Day-one public statement:** {incident.day_one_statement or 'Not yet issued'}",
"",
"## Resolution",
"",
f"**Root cause:** {incident.root_cause or 'Under investigation'}",
f"**Permanent fix:** {incident.permanent_fix or 'Pending'}",
f"**Public follow-up:** {incident.public_followup or 'Pending — must match visibility of original incident'}",
"",
f"**Ready for a public statement?** {'Yes — containment verified' if incident.is_ready_for_public_statement() else 'No — containment not yet verified, hold public statement to a factual acknowledgment only'}",
]
return "\n".join(lines)
# ── Worked example ───────────────────────────────────────────────────────────
# The prescription-drug-interaction chatbot scenario from the opening.
def build_example() -> IncidentTimeline:
incident = IncidentTimeline(
incident_name="Chatbot Drug-Interaction Misinformation",
description=(
"Support chatbot told a user a prescription drug was safe to combine "
"with alcohol; the interaction is actually dangerous. Screenshot "
"circulating publicly with rapid share growth."
),
severity=IncidentSeverity.CRITICAL,
)
incident.containment_action = "Disabled chatbot medication-interaction responses; routed all drug-related queries to a static safety page and human support"
incident.containment_status = ContainmentStatus.VERIFIED
incident.add_notification("Engineering lead", "T+10min", "Reproducibility check requested; asked for immediate rollback option")
incident.add_notification("Legal / Trust & Safety", "T+20min", "Facts as known; asked whether this triggers a regulatory incident-reporting obligation")
incident.add_notification("Executive stakeholders", "T+35min", "One-paragraph factual summary; no fix timeline committed yet")
incident.add_notification("Support / social team", "T+40min", "Holding statement provided for consistent external responses")
incident.holding_statement = (
"We're aware of a report that our support assistant gave inaccurate "
"medication safety information. We've disabled AI-generated responses "
"for medication questions while we investigate, and are routing these "
"questions to our human support team and official safety resources."
)
incident.day_one_statement = (
"Yesterday, our support assistant gave a user inaccurate information "
"about a medication interaction. We disabled AI responses for "
"medication-related questions within the hour and are routing them to "
"trained support staff. We are investigating the root cause and will "
"share what changes we're making before this feature is re-enabled."
)
return incident
if __name__ == "__main__":
incident = build_example()
print(render_incident_log(incident))
print()
score, gaps = readiness_check(
has_defined_containment_levers=True,
has_defined_escalation_order=True,
has_holding_statement_template=False,
has_named_incident_owner=True,
has_postmortem_process=False,
)
print(f"Pre-incident readiness score: {score}/5")
for gap in gaps:
print(f" GAP: {gap}")
print()
print(
"Takeaway: the day-one statement commits to no fix date, only a "
"commitment to investigate and share changes before re-enabling — "
"that's the difference between a statement the team can actually "
"keep and one that creates a second incident when it slips."
)