Phase 7: Responsible AI, Governance & Risk · 140 min · Python · EU AI Act · NIST AI RMF

Project: Responsible AI Review

Four lessons, one feature, one document a launch review would actually accept.

Real-world application: A complete, coherent responsible AI review — consequence-scan, disclosure plan, regulatory triage, incident readiness, and guardrail decisions for one real feature — is precisely the artifact research identifies as the highest-leverage AI PM portfolio piece: it demonstrates technical fluency, judgment under uncertainty, and the ability to ship consequential features responsibly, all against a concrete example rather than in the abstract. This project also feeds directly into the Phase 9 capstone case study.

What you will learn

Assemble a consequence-scan, ethical KPIs, and a product-facing system card for the same feature carried through Projects 1-3
Classify that feature under the EU AI Act and map it to NIST AI RMF actions, translating the result into PRD-ready requirements
Write an incident-readiness plan and a guardrail decision set for the feature's most plausible failure and abuse scenarios
Produce a single, complete responsible AI review document a real launch review would accept as sufficient
Explicitly connect this review's findings back to the opportunity assessment, PRD, and eval plan from Projects 1-3

The Problem

Picture the launch review you'd actually sit in for the feature you've been carrying through this course. The opportunity assessment from Phase 2 said this was worth building. The PRD from Phase 4 said exactly what it should do, including its guardrails and cost model. The evaluation plan from Phase 5 said how you'd know if it's working. Now someone at the table — legal, a skeptical VP, a trust-and-safety lead who has seen a feature like this go wrong before — asks the question none of those three documents fully answers: if this goes wrong, what happens, and are we ready?

That question doesn't get a good answer from four separate documents stapled together. It gets a good answer from one responsible AI review that pulls the actual analysis from this phase's five lessons — consequence-scan, disclosure plan, regulatory classification, incident readiness, guardrail decisions — and applies every one of them to the same feature you've been developing since Phase 2. This project is that document. It's also, not incidentally, the fourth of four project artifacts the Phase 9 capstone will assemble into a single portfolio case study, so the discipline of writing this cleanly now pays off twice: once in this phase, and again when you're building the artifact a hiring manager actually reads.

What this project assembles

Rather than introducing new frameworks, this project is deliberately an assembly exercise — the hard thinking already happened in Lessons 1-5. Your job is to apply five completed frameworks to one real feature and make them cohere into a single narrative, not five disconnected exercises:

Consequence-scan and ethical KPIs (Lesson 1) — who is affected if your feature's AI component is wrong, and what fairness metric with what threshold would catch it
Product-facing disclosure (Lesson 2) — what a user or affected person needs to know about this specific feature, distinct from any internal model documentation
Regulatory classification (Lesson 3) — the feature's EU AI Act tier and the NIST AI RMF actions that follow from it, turned into requirements a PRD could actually contain
Incident readiness (Lesson 4) — the single most plausible way this feature fails publicly, and whether your team could actually execute the first-hour/first-day response if it happened tomorrow
Guardrail decisions (Lesson 5) — the two or three most plausible abuse cases for this specific feature, each with an allow/deflect/escalate/refuse decision and a stated friction tradeoff

Continuity is the point, not a formality
If your Phase 2 opportunity assessment picked "AI-assisted loan pre-approval" and your Phase 4 PRD specified how it works, this review has to be about that feature — not a generic loan-AI risk essay. A reviewer (and, later, an interviewer looking at your portfolio) can tell immediately when a "risk assessment" was written in isolation versus written against a real PRD's actual design decisions. Reuse your own specifics: the affected groups you already named, the guardrails you already specified, the cost/latency tradeoffs that already constrain what containment options are realistic.

Why this order, and why it's not five separate documents

Notice the dependency chain: the consequence-scan's affected groups inform what the disclosure needs to say to whom. The regulatory tier tells you whether "disclosure" is a nice-to-have or a legal floor. The incident-readiness plan only makes sense once you know which failure mode is most plausible — which the consequence-scan already told you. And the guardrail decisions are where the abuse cases from the security lesson meet the same severity/reversibility logic from the consequence-scan in Lesson 1. Writing these as five unconnected sections misses the actual skill this project is testing: can you hold the whole risk picture for one feature in your head at once, the way a real responsible-AI review requires, instead of running five checklists independently and hoping they add up.

Your Phase 2-4 feature is an AI resume-screening tool for internal recruiter use, with a human recruiter reviewing every AI-generated shortlist before any candidate is contacted. Your consequence-scan (Lesson 1) already found a plausible false-positive-parity gap for candidates with employment gaps. How should that finding shape your regulatory classification and incident-readiness sections in this review?

This is exactly the connective tissue the project is testing. The regulatory tier comes from the use-case domain (employment, an Annex III high-risk category) independent of any specific finding, but a known, plausible fairness gap is precisely the kind of finding that should drive your incident-readiness section's most-plausible-failure scenario — if it's the failure mode you already know is most likely, your first-hour containment and holding-statement plan should be written with that specific scenario in mind, not a generic one. Carrying the finding forward across sections, rather than treating each lesson's exercise as disposable, is the actual skill this project builds.

What makes this review launch-ready, not just complete

A launch review doesn't want five checklists marked done — it wants a decision it can act on. That means your review needs to end with the same kind of explicit call Lesson 1's ship_decision() produces: ship, mitigate, or redesign, stated once for the whole feature, with the strongest reason for that call named explicitly (not the average of five separate green checkmarks). If your regulatory tier is High and your consequence-scan found a real, unaddressed fairness gap, the honest answer might be "mitigate before launch, specifically: close the false-positive gap to under 5 points and add the human-oversight logging the Lesson 3 triage flagged as missing" — a decision with two concrete, checkable conditions, not a vague "proceed with caution."

Your assembled review shows: all ethical KPIs passing, a Minimal EU AI Act tier, but an incident-readiness score of 1/5 because no one has ever discussed what would happen if this feature failed publicly. What should the overall recommendation logic conclude, and why does that matter even though nothing else in the review is flagged?

Passing KPIs and a low regulatory tier lower the probability of a public failure, but they don't make the probability zero, and low incident readiness means the team would be improvising its first hour in real time if something did go wrong — exactly the failure mode Lesson 4 opens with. The right call isn't to ignore this gap because other sections look clean, and it isn't to redesign the whole feature over a process gap rather than a design flaw — it's to treat minimal incident readiness as a real, fixable mitigation item (write the holding-statement template, name an incident owner) before calling the feature fully launch-ready.

Build

Build it

The code artifact lesson_07_06_responsible_ai_review.py assembles the five lessons' data models into one review:

Reuses ConsequenceScan, EthicalKPI, ProductSystemCard, AISystemProfile, IncidentTimeline, and AbuseCase/GuardrailDecision structures (simplified, self-contained versions) from Lessons 1-5
ResponsibleAIReview — the top-level container tying all five pieces to one feature name and description
overall_recommendation() — combines the consequence-scan ship-decision logic, the regulatory tier, and any failing guardrail/KPI checks into one final SHIP/MITIGATE/REDESIGN call for the whole feature
render_full_review() — produces the complete markdown responsible AI review document
A worked example: an AI-assisted loan pre-approval feature (consistent with the running example from Lessons 1 and 3), assembled end to end

"""
lesson_07_06_responsible_ai_review.py

Project: Responsible AI Review. Assembles the consequence-scan, ethical
KPIs, disclosure/system-card summary, EU AI Act / NIST AI RMF
classification, incident readiness, and guardrail decisions from
Lessons 1-5 into one complete, launch-ready responsible AI review for a
single feature — the fourth project artifact in the course-long thread,
feeding the Phase 9 capstone case study.

References: ProdPad (https://www.prodpad.com/blog/ethics-in-ai/),
EU AI Act (https://artificialintelligenceact.eu/),
NIST AI RMF (https://www.nist.gov/itl/ai-risk-management-framework),
Fairlearn (https://fairlearn.org/).

Usage:
    python lesson_07_06_responsible_ai_review.py

No external dependencies, no network calls, no API keys required.
"""

from __future__ import annotations

from dataclasses import dataclass, field
from enum import Enum

# ── Lesson 1: consequence-scan and ethical KPIs ────────────────────────────

class Severity(Enum):
    LOW = "Low"
    MEDIUM = "Medium"
    HIGH = "High"
    SEVERE = "Severe"

class Reversibility(Enum):
    EASY = "Easy"
    MODERATE = "Moderate"
    HARD = "Hard"

@dataclass
class AffectedGroup:
    name: str
    how_affected: str
    severity: Severity
    reversibility: Reversibility

@dataclass
class EthicalKPI:
    name: str
    group_values: dict[str, float]
    max_allowed_gap: float

    def observed_gap(self) -> float:
        values = list(self.group_values.values())
        return max(values) - min(values) if values else 0.0

    def passes(self) -> bool:
        return self.observed_gap() <= self.max_allowed_gap

# ── Lesson 2: product-facing disclosure ────────────────────────────────────

@dataclass
class DisclosureSummary:
    capability_summary: str
    known_limitations: list[str]
    appeal_path: str
    oversight_level: str

# ── Lesson 3: regulatory classification ────────────────────────────────────

class EUTier(Enum):
    UNACCEPTABLE = "Unacceptable"
    HIGH = "High"
    LIMITED = "Limited"
    MINIMAL = "Minimal"

@dataclass
class RegulatoryClassification:
    tier: EUTier
    triggering_rule: str
    key_obligations: list[str]
    human_oversight_meaningfully_exercised: bool

# ── Lesson 4: incident readiness ───────────────────────────────────────────

@dataclass
class IncidentReadiness:
    most_plausible_failure: str
    containment_action: str
    holding_statement: str
    readiness_score_out_of_5: int

# ── Lesson 5: guardrail decisions ──────────────────────────────────────────

@dataclass
class AbuseCase:
    name: str
    severity: Severity
    legitimate_use_exists: bool

@dataclass
class GuardrailDecision:
    abuse_case: AbuseCase
    response: str  # "Allow" | "Deflect" | "Escalate" | "Refuse"
    rationale: str

# ── Assembled review ────────────────────────────────────────────────────────

@dataclass
class ResponsibleAIReview:
    feature_name: str
    feature_description: str
    affected_groups: list[AffectedGroup]
    ethical_kpis: list[EthicalKPI]
    disclosure: DisclosureSummary
    regulatory: RegulatoryClassification
    incident_readiness: IncidentReadiness
    guardrail_decisions: list[GuardrailDecision]

def overall_recommendation(review: ResponsibleAIReview) -> tuple[str, list[str]]:
    """Combines all five sections into one SHIP / MITIGATE / REDESIGN call."""
    reasons: list[str] = []
    failing_kpis = [k for k in review.ethical_kpis if not k.passes()]
    if failing_kpis:
        reasons.append(
            f"{len(failing_kpis)} ethical KPI(s) failing target gap: "
            + ", ".join(k.name for k in failing_kpis)
        )

    hard_to_reverse_high_severity = any(
        g.severity in (Severity.HIGH, Severity.SEVERE) and g.reversibility == Reversibility.HARD
        for g in review.affected_groups
    )
    if hard_to_reverse_high_severity:
        reasons.append("At least one affected group faces high/severe, hard-to-reverse harm")

    if review.regulatory.tier == EUTier.HIGH and not review.regulatory.human_oversight_meaningfully_exercised:
        reasons.append("High-risk EU AI Act tier with human oversight not meaningfully exercised (rubber-stamp risk)")

    if review.incident_readiness.readiness_score_out_of_5 < 3:
        reasons.append(
            f"Incident readiness is low ({review.incident_readiness.readiness_score_out_of_5}/5) "
            "for this feature's most plausible failure mode"
        )

    refusing_or_escalating_unaddressed = [
        d for d in review.guardrail_decisions
        if d.response == "Refuse" and d.abuse_case.severity == Severity.SEVERE
    ]

    if not reasons:
        return "SHIP", ["All KPIs pass, no hard-to-reverse high-severity gaps, oversight is real, incident readiness is adequate."]

    if hard_to_reverse_high_severity or (
        review.regulatory.tier == EUTier.HIGH and not review.regulatory.human_oversight_meaningfully_exercised
    ):
        return "REDESIGN", reasons

    return "MITIGATE", reasons

def render_full_review(review: ResponsibleAIReview) -> str:
    decision, reasons = overall_recommendation(review)
    lines = [
        f"# Responsible AI Review — {review.feature_name}",
        "",
        review.feature_description,
        "",
        "## 1. Consequence Scan & Ethical KPIs",
        "",
        "| Affected group | How affected | Severity | Reversibility |",
        "|---|---|---|---|",
    ]
    for g in review.affected_groups:
        lines.append(f"| {g.name} | {g.how_affected} | {g.severity.value} | {g.reversibility.value} |")
    lines += ["", "| Ethical KPI | Observed gap | Max allowed | Pass? |", "|---|---|---|---|"]
    for k in review.ethical_kpis:
        status = "PASS" if k.passes() else "FAIL"
        lines.append(f"| {k.name} | {k.observed_gap():.1%} | {k.max_allowed_gap:.0%} | {status} |")

    lines += [
        "",
        "## 2. Product-Facing Disclosure",
        "",
        review.disclosure.capability_summary,
        "",
        "Known limitations:",
    ]
    lines += [f"- {lim}" for lim in review.disclosure.known_limitations]
    lines += [
        "",
        f"Oversight level: {review.disclosure.oversight_level}",
        f"Appeal path: {review.disclosure.appeal_path}",
        "",
        "## 3. Regulatory Classification",
        "",
        f"**EU AI Act tier: {review.regulatory.tier.value}** — {review.regulatory.triggering_rule}",
        "",
        "Key obligations:",
    ]
    lines += [f"- {o}" for o in review.regulatory.key_obligations]
    lines += [
        "",
        f"Human oversight meaningfully exercised: {'Yes' if review.regulatory.human_oversight_meaningfully_exercised else 'No — flagged as a gap'}",
        "",
        "## 4. Incident Readiness",
        "",
        f"Most plausible failure mode: {review.incident_readiness.most_plausible_failure}",
        f"Containment action: {review.incident_readiness.containment_action}",
        f"Holding statement (drafted in advance): {review.incident_readiness.holding_statement}",
        f"Readiness score: {review.incident_readiness.readiness_score_out_of_5}/5",
        "",
        "## 5. Guardrail Decisions",
        "",
        "| Abuse case | Severity | Legitimate use? | Decision | Rationale |",
        "|---|---|---|---|---|",
    ]
    for d in review.guardrail_decisions:
        c = d.abuse_case
        lines.append(
            f"| {c.name} | {c.severity.value} | {'Yes' if c.legitimate_use_exists else 'No'} | {d.response} | {d.rationale} |"
        )

    lines += [
        "",
        f"## Overall Recommendation: {decision}",
        "",
    ]
    lines += [f"- {r}" for r in reasons]
    return "\n".join(lines)

# ── Worked example ───────────────────────────────────────────────────────────
# The AI loan pre-approval feature, consistent with the running example
# used in Lessons 1 and 3 of this phase.

def build_example() -> ResponsibleAIReview:
    return ResponsibleAIReview(
        feature_name="AI Loan Pre-Approval Assistant",
        feature_description=(
            "Model scores loan applications and auto-denies pre-approval below "
            "a confidence threshold; approved applications route to a human "
            "underwriter for final sign-off."
        ),
        affected_groups=[
            AffectedGroup(
                "Self-employed applicants",
                "Irregular income documentation reads as risk signal",
                Severity.HIGH,
                Reversibility.MODERATE,
            ),
            AffectedGroup(
                "First-time applicants (thin credit file)",
                "Sparse credit history scored similarly to poor credit history",
                Severity.MEDIUM,
                Reversibility.EASY,
            ),
        ],
        ethical_kpis=[
            EthicalKPI(
                "False-positive parity (wrongly denied pre-approval)",
                {"Salaried applicants": 0.04, "Self-employed applicants": 0.13},
                0.05,
            ),
        ],
        disclosure=DisclosureSummary(
            capability_summary="Estimates loan pre-approval likelihood; a human underwriter makes the final call.",
            known_limitations=[
                "May be less accurate for applicants with non-traditional income documentation",
                "Pre-approval denial is not final — appeal routes to a human underwriter",
            ],
            appeal_path="Denied applicants can request human underwriter review within 10 business days.",
            oversight_level="Human-reviewed before any final decision is issued",
        ),
        regulatory=RegulatoryClassification(
            tier=EUTier.HIGH,
            triggering_rule="Annex III domain: access to essential private services (credit)",
            key_obligations=[
                "Risk management system maintained for the system's lifecycle",
                "Human oversight mechanism genuinely exercised, not nominal",
                "Post-market monitoring plan with defined review cadence",
            ],
            human_oversight_meaningfully_exercised=False,
        ),
        incident_readiness=IncidentReadiness(
            most_plausible_failure="A journalist finds the self-employed false-positive gap and reports it publicly",
            containment_action="Route all self-employed applicant denials to mandatory human review pending gap closure",
            holding_statement="We are aware of a reported disparity in pre-approval outcomes and are reviewing our process, effective immediately for affected applicants.",
            readiness_score_out_of_5=2,
        ),
        guardrail_decisions=[
            GuardrailDecision(
                AbuseCase("Applicant submits fabricated income documents to game the model", Severity.HIGH, False),
                "Escalate",
                "Flag inconsistent documentation patterns for human underwriter review rather than auto-approving or auto-denying",
            ),
        ],
    )

if __name__ == "__main__":
    review = build_example()
    print(render_full_review(review))
    print()
    print(
        "Takeaway: the overall recommendation is REDESIGN, driven by specific, "
        "checkable facts — a failing false-positive-parity KPI, a High-risk "
        "tier with oversight that isn't meaningfully exercised, and low "
        "incident readiness — not a vague sense of caution. That's what makes "
        "this a launch review can act on instead of another document to file away."
    )

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Check Yourself — plus a hands-on lab, quiz, and project artifact.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card.