Phase 7: Responsible AI, Governance & Risk · 40 min · Model/system cards · Python

Transparency & Explainability

The ML team's model card is written for the next engineer. Your system card is written for the person it affects.

What you will learn

Decide what to disclose to users about an AI feature, and when disclosure is required versus optional
Distinguish a product-facing system card from an ML team's internal model card
Write disclosure language that is honest, specific, and doesn't create unnecessary alarm or false reassurance
Produce a complete product-facing system card for a real AI feature

The Problem

A healthcare scheduling startup launches an AI feature that predicts which patients are likely to no-show an appointment and proactively double-books their slot. It works — no-show losses drop 30%. Then a patient who showed up on time, only to be told their appointment was given away because the algorithm predicted they wouldn't come, posts about it. The company's engineering team has a thorough internal model card: training data, feature list, AUC, calibration plots. None of it says one sentence in language the patient could have understood before it happened to them: "We predict no-show risk and may adjust scheduling accordingly."

The model card wasn't wrong. It was written for the wrong audience. A model card documents a model for the people who build and maintain it — data scientists, ML engineers, auditors. It answers "what data, what architecture, what known limitations." A product-facing system card answers a completely different question: what does a person interacting with or affected by this system need to know, in language they can act on? Conflating the two — assuming the internal model card "covers" transparency because it exists — is the actual root cause of the story above, and it's an extremely common one.

What to disclose, and when

Not every AI feature needs the same level of disclosure. A useful way to calibrate: disclosure obligation scales with consequence and surprise, not with technical sophistication.

Always disclose the AI is present when a reasonable person would otherwise assume they're interacting with a human, or when the output looks like it could be human-authored (chat, generated summaries, synthetic voice). This is close to a legal floor in many jurisdictions now (see Lesson 3), not just best practice.
Disclose the mechanism when a decision is consequential — not the model architecture, but what the system does and doesn't do. "We use an automated system to flag applications for additional review" is disclosure. "We use a gradient-boosted ensemble trained on 40 features" is not disclosure, it's noise that satisfies no one.
Disclose limitations that change user behavior appropriately — if a system can be confidently wrong (most generative AI features), users need to know that before they rely on it, not as a buried terms-of-service clause. "This summary may omit or misstate details — verify anything you'll act on" is a limitation disclosure that actually changes how a careful user behaves.
Disclose an appeal or override path if one exists — and if the consequence is high enough (per Lesson 1's severity/reversibility framework), the absence of an appeal path is itself something to disclose, or better, something to build before launch.

Disclosure is not a liability shield if the underlying decision is unfair
A well-written disclosure ("this recommendation is generated by AI and may be imperfect") does not fix a false-positive-parity gap from Lesson 1 — it just tells the affected person the system might be wrong, without telling them it's disproportionately wrong about people like them. Regulators, journalists, and increasingly your own users treat "we disclosed it" and "we disclosed it clearly enough to matter" as different bars. Transparency is a complement to responsible design, not a substitute for it.

The product-facing system card

A product-facing system card is a short document — one to two pages, written for a person who is not an ML engineer — that a PM owns and a legal/trust-and-safety reviewer signs off on. It typically covers:

What the system does — plain-language description of the capability, not the architecture
What it doesn't do / known limitations — the honest failure modes: it can be confidently wrong, it doesn't have real-time information, it was trained predominantly on X kind of data and may perform worse for Y
How outputs are used — is this fully automated, does a human review it, can the user override it
What data it uses about the user — at a level a privacy-conscious user actually wants answered
How to get help or appeal — the concrete path if the system gets something wrong for you specifically
Last reviewed date — system cards go stale exactly like models do; a card without a review date is a card nobody's kept honest

This is meaningfully different from an internal model card, which would instead cover training data provenance, evaluation metrics, intended use restrictions for other engineering teams, and versioning — useful, but not written in language a user or a journalist would accept as a real answer.

A team ships an AI resume-screening tool for internal recruiter use only — it never faces external candidates directly, and a human recruiter makes every final decision after reviewing the AI's shortlist. Does this feature need a product-facing system card, and if so, who is the "product" audience?

"Product-facing" doesn't mean "external users only" — it means whoever the direct users and consequentially-affected people are. Here, recruiters are the direct users and need a system card written for them: what the tool does, its known limitations (e.g. from Lesson 1's consequence-scan), and what to do if they suspect it screened someone out incorrectly. Candidates are affected but aren't the direct interface audience; a separate, shorter disclosure to candidates about automated screening being used may still be legally required depending on jurisdiction, but the system card itself is written for the recruiter who has to use the tool responsibly.

Writing disclosure language that actually works

The most common failure in disclosure writing is picking one of two bad defaults: legally over-hedged boilerplate ("outputs may vary and are provided as-is without warranty of any kind") that satisfies a lawyer and informs no one, or breezy marketing copy that undersells real limitations. The target is specific, honest, and short: state what the system does, state the one or two ways it's most likely to be wrong for this use case specifically, and state what to do if it is. A good test: would a skeptical user read this and know something actionable they didn't know before? If not, it's not disclosure, it's decoration.

A team drafts disclosure language for a generative AI meeting-summary feature: "This output was generated using advanced machine learning technology and may not be 100% accurate in all circumstances." A PM reviewing it flags it as insufficient. What's the specific problem, and what should replace it?

This is the exact failure mode called out in this lesson: vague, legally-safe language ("advanced machine learning technology," "may not be 100% accurate in all circumstances") passes a legal review but fails the actual test of disclosure — would a skeptical user learn something actionable? Naming the technology by brand isn't the issue and isn't a legal requirement here. The fix is specificity: name the actual likely failure mode for this feature (details omitted or misstated, especially for messy audio) and state a concrete action the user should take (verify before forwarding). That's the difference between decoration and disclosure this lesson describes.

Build

Build it

The code artifact lesson_07_02_product_system_card.py implements a product-facing system card generator:

SystemCapability — what the system does, in plain language
KnownLimitation — a specific failure mode and how a user should account for it
ProductSystemCard — the full structured card: capability, limitations, human oversight level, data usage, appeal path, review date
render_system_card() — produces the markdown card, plus a companion internal-vs-external comparison note
A worked example: the no-show-prediction scheduling feature from the opening scenario, written as it should have shipped

"""
lesson_07_02_product_system_card.py

A product-facing system card generator, distinct from an internal ML
model card. Produces a short, plain-language document describing what
an AI system does, its known limitations, how much human oversight is
involved, what user data it touches, and how to appeal or get help —
the artifact a PM owns and ships alongside a consequential AI feature.

Reference: ProdPad (https://www.prodpad.com/blog/ethics-in-ai/).

Usage:
    python lesson_07_02_product_system_card.py

No external dependencies, no network calls, no API keys required.
"""

from __future__ import annotations

from dataclasses import dataclass, field
from enum import Enum

class OversightLevel(Enum):
    FULLY_AUTOMATED = "Fully automated — no human reviews individual outputs"
    HUMAN_REVIEWED = "Human-reviewed — a person reviews the AI's output before it takes effect"
    HUMAN_OVERRIDABLE = "Human-overridable — AI acts automatically but a person can reverse it"
    ADVISORY_ONLY = "Advisory only — AI suggests, a human decides"

@dataclass
class SystemCapability:
    plain_language_description: str
    example_output: str

@dataclass
class KnownLimitation:
    description: str
    how_user_should_account_for_it: str

@dataclass
class ProductSystemCard:
    feature_name: str
    capability: SystemCapability
    limitations: list[KnownLimitation]
    oversight_level: OversightLevel
    user_data_used: list[str]
    appeal_path: str
    last_reviewed: str
    audience: str = "end users"

    def is_stale(self, current_date: str) -> bool:
        # Simple lexicographic comparison assumes ISO 8601 (YYYY-MM-DD) dates.
        return self.last_reviewed < current_date

def render_system_card(card: ProductSystemCard) -> str:
    lines = [
        f"# System Card: {card.feature_name}",
        "",
        f"*Audience: {card.audience} — this is a product-facing card, not the internal ML model card.*",
        "",
        "## What this system does",
        "",
        card.capability.plain_language_description,
        "",
        f"Example: {card.capability.example_output}",
        "",
        "## What it doesn't do / known limitations",
        "",
    ]
    for lim in card.limitations:
        lines.append(f"- **{lim.description}** — {lim.how_user_should_account_for_it}")

    lines += [
        "",
        "## How outputs are used",
        "",
        card.oversight_level.value,
        "",
        "## What data this system uses about you",
        "",
    ]
    for d in card.user_data_used:
        lines.append(f"- {d}")

    lines += [
        "",
        "## If something seems wrong",
        "",
        card.appeal_path,
        "",
        f"*Last reviewed: {card.last_reviewed}*",
    ]
    return "\n".join(lines)

# ── Worked example ───────────────────────────────────────────────────────────
# The no-show-prediction scheduling feature from the lesson's opening
# scenario, written the way it should have shipped in the first place.

def build_example() -> ProductSystemCard:
    return ProductSystemCard(
        feature_name="No-Show Risk Scheduling Assistant",
        capability=SystemCapability(
            plain_language_description=(
                "This system estimates the likelihood that a scheduled "
                "appointment will be missed, based on past scheduling "
                "patterns. When risk is predicted to be high, the system "
                "may offer your time slot to another patient as a backup."
            ),
            example_output="Predicted no-show risk: High -> slot flagged for possible double-booking.",
        ),
        limitations=[
            KnownLimitation(
                description="The prediction can be wrong for a specific appointment, even when it's usually accurate in aggregate.",
                how_user_should_account_for_it=(
                    "If you intend to keep your appointment, confirm it through the "
                    "reminder link at least 24 hours ahead — confirmed appointments "
                    "are not double-booked."
                ),
            ),
            KnownLimitation(
                description="The model was trained primarily on daytime weekday appointments and is less tested on evening/weekend slots.",
                how_user_should_account_for_it=(
                    "Evening and weekend appointments are currently excluded from "
                    "double-booking regardless of predicted risk."
                ),
            ),
        ],
        oversight_level=OversightLevel.HUMAN_OVERRIDABLE,
        user_data_used=[
            "Your appointment history at this practice (attendance, cancellations, reschedules)",
            "Appointment type and time slot",
            "This system does not use your medical record contents",
        ],
        appeal_path=(
            "If your confirmed appointment was still double-booked, contact the "
            "front desk immediately — confirmed appointments are prioritized and "
            "this is treated as a system error to be corrected same-day."
        ),
        last_reviewed="2026-05-01",
    )

if __name__ == "__main__":
    card = build_example()
    print(render_system_card(card))
    print()
    print(f"Stale as of 2026-11-01? {card.is_stale('2026-11-01')}")
    print(
        "Takeaway: notice what's absent — no model architecture, no AUC, no "
        "feature list. Those belong in the internal model card. This card "
        "answers exactly the question the patient in the opening scenario "
        "needed answered before it happened to them."
    )

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Check Yourself — plus a hands-on lab, quiz, and project artifact.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card.