Phase 8: Evaluation, Safety & Observability · 50 min · Hugging Face Model Cards · MLflow · Jinja2

Model Cards: Documenting ML Models for Transparency and Compliance

If you can't document it, you don't understand it. If you don't document it, nobody trusts it.

Real-world application: Model cards are increasingly required by regulation (EU AI Act) and internal governance. Engineers who can produce proper model documentation are qualified for ML Engineer roles at companies with mature ML practices.

What you will learn

Understand what model cards are and why they're now legally required in some jurisdictions
Write a complete model card following the Mitchell et al. framework
Document model performance across demographic subgroups
Identify ethical considerations, limitations, and recommended use cases
Generate model cards programmatically from training metadata

The Problem

A hospital deploys a model that predicts patient readmission risk. Doctors use it to decide which patients get extra follow-up care. Six months later, a study finds the model systematically underestimates risk for Black patients — they're denied follow-up care at higher rates than white patients with the same clinical profile.

The hospital didn't know because:

No one documented what population the model was trained on
No one tested performance across demographic subgroups
No one wrote down the model's intended use cases or limitations
No one created a model card

This actually happened. The 2019 Obermeyer et al. study found a widely-used healthcare algorithm discriminated against Black patients, affecting 200M people. The algorithm had no model card, no subgroup analysis, no documented limitations.

The Concept

What Is a Model Card?

A model card is a structured documentation file that accompanies a machine learning model. It answers:

What does this model do? (intended use, task, output)
How was it trained? (data, algorithm, training process)
How well does it work? (metrics, subgroup performance, eval conditions)
When should you NOT use it? (limitations, out-of-scope uses, ethical risks)
Who is responsible? (authors, contact, maintenance)

Model cards were proposed by Mitchell et al. (2019) and have become the industry standard, adopted by Hugging Face, Google, TensorFlow Hub, and required by the EU AI Act for high-risk AI systems.

The Model Card Template

# Model Card: [Model Name]

## Model Details
- **Model type:** [e.g., Fine-tuned BERT, Random Forest, CNN]
- **Version:** [e.g., 1.2.0]
- **Date trained:** [2026-07-15]
- **Authors:** [names/teams]
- **Contact:** [email or issue tracker]
- **License:** [MIT, Apache 2.0, proprietary]

## Intended Use
- **Primary intended uses:** [What the model is designed for]
- **Primary intended users:** [Who should use it]
- **Out-of-scope uses:** [What the model should NOT be used for]

## Training Data
- **Source:** [Where the data came from]
- **Size:** [Number of samples]
- **Demographics:** [Distribution across relevant groups]
- **Preprocessing:** [What transformations were applied]
- **Known biases in data:** [Sampling biases, historical biases]

## Evaluation Data
- **Source:** [Hold-out test set, external benchmark]
- **Size:** [Number of samples]
- **Demographics:** [Distribution across relevant groups]
- **Difference from training data:** [Distribution shift, if any]

## Metrics
- **Metrics selected:** [Accuracy, F1, AUC, etc. — and why]
- **Thresholds:** [Decision thresholds and their impact]
- **Overall performance:** [Aggregate metrics]

## Subgroup Performance
| Group | Accuracy | F1 | False Positive Rate | False Negative Rate |
|-------|----------|-----|---------------------|---------------------|
| Group A | 0.92 | 0.89 | 0.05 | 0.08 |
| Group B | 0.87 | 0.82 | 0.12 | 0.15 |
| Overall | 0.90 | 0.86 | 0.08 | 0.11 |

## Ethical Considerations
- [Potential for harm, bias, misuse]
- [Mitigations applied]

## Limitations
- [What the model can't do]
- [Where it may fail]
- [Performance degradation conditions]

## Recommendations
- [How to deploy safely]
- [What to monitor in production]
- [When to retrain]

Your team trains a sentiment analysis model on English Twitter data. A product manager wants to deploy it for analyzing customer support tickets in Japanese. The model card says: "Training data: 2M English-language tweets from 2024-2025. Out-of-scope uses: Non-English text." What should happen?

Model cards exist precisely for this scenario. The 'out-of-scope uses' section is a formal boundary that protects against misuse. A model trained on English Twitter data has never seen Japanese text — its embeddings, tokenization, and learned patterns are English-specific. Deploying it on Japanese input would produce garbage predictions that could mislead business decisions. The correct action: either train a new model on Japanese data, or use a multilingual model with its own model card that covers Japanese.

Subgroup Analysis: The Most Important Section

The subgroup performance table is the section that catches discrimination. If your model achieves 90% accuracy overall but 70% accuracy for one demographic group, the aggregate metric hides the harm.

import pandas as pd
from sklearn.metrics import accuracy_score, precision_score, recall_score

def subgroup_analysis(y_true, y_pred, groups, group_names):
    """Generate subgroup performance table for model card."""
    results = []
    
    for group_val, group_name in zip(groups.unique(), group_names):
        mask = groups == group_val
        if mask.sum() == 0:
            continue
        
        y_true_g = y_true[mask]
        y_pred_g = y_pred[mask]
        
        results.append({
            "group": group_name,
            "n": int(mask.sum()),
            "accuracy": accuracy_score(y_true_g, y_pred_g),
            "precision": precision_score(y_true_g, y_pred_g, average='weighted', zero_division=0),
            "recall": recall_score(y_true_g, y_pred_g, average='weighted', zero_division=0),
            "false_positive_rate": ((y_pred_g == 1) & (y_true_g == 0)).sum() / max(((y_true_g == 0).sum()), 1),
            "false_negative_rate": ((y_pred_g == 0) & (y_true_g == 1)).sum() / max(((y_true_g == 1).sum()), 1),
        })
    
    # Add overall row
    results.append({
        "group": "Overall",
        "n": len(y_true),
        "accuracy": accuracy_score(y_true, y_pred),
        "precision": precision_score(y_true, y_pred, average='weighted', zero_division=0),
        "recall": recall_score(y_true, y_pred, average='weighted', zero_division=0),
        "false_positive_rate": ((y_pred == 1) & (y_true == 0)).sum() / max(((y_true == 0).sum()), 1),
        "false_negative_rate": ((y_pred == 0) & (y_true == 1)).sum() / max(((y_true == 1).sum()), 1),
    })
    
    return pd.DataFrame(results)

# Example: Detecting bias in a hiring model
df = pd.DataFrame({
    "y_true": [...],
    "y_pred": [...],
    "gender": [...],   # male, female, non-binary
    "age_group": [...], # 20-30, 30-40, 40-50, 50+
})

# Check: does the model perform equally across gender?
print(subgroup_analysis(df.y_true, df.y_pred, df.gender, ["Male", "Female", "Non-binary"]))

Programmatic Model Card Generation

from dataclasses import dataclass, asdict
from datetime import datetime
import json

@dataclass
class ModelCard:
    # Model details
    model_name: str
    model_type: str
    version: str
    date_trained: str
    authors: list[str]
    license: str
    
    # Intended use
    intended_uses: list[str]
    intended_users: list[str]
    out_of_scope_uses: list[str]
    
    # Training data
    training_data_source: str
    training_data_size: int
    training_data_demographics: dict
    preprocessing_steps: list[str]
    known_biases: list[str]
    
    # Evaluation
    eval_data_source: str
    eval_data_size: int
    metrics: dict
    subgroup_performance: list[dict]
    
    # Ethics and limitations
    ethical_considerations: list[str]
    limitations: list[str]
    recommendations: list[str]
    
    def to_markdown(self) -> str:
        """Generate markdown model card."""
        md = f"# Model Card: {self.model_name}\n\n"
        md += f"## Model Details\n"
        md += f"- **Model type:** {self.model_type}\n"
        md += f"- **Version:** {self.version}\n"
        md += f"- **Date trained:** {self.date_trained}\n"
        md += f"- **Authors:** {', '.join(self.authors)}\n"
        md += f"- **License:** {self.license}\n\n"
        
        md += f"## Intended Use\n"
        md += f"- **Primary intended uses:** {'; '.join(self.intended_uses)}\n"
        md += f"- **Primary intended users:** {'; '.join(self.intended_users)}\n"
        md += f"- **Out-of-scope uses:** {'; '.join(self.out_of_scope_uses)}\n\n"
        
        md += f"## Training Data\n"
        md += f"- **Source:** {self.training_data_source}\n"
        md += f"- **Size:** {self.training_data_size:,} samples\n"
        md += f"- **Demographics:** {json.dumps(self.training_data_demographics, indent=2)}\n"
        md += f"- **Preprocessing:** {'; '.join(self.preprocessing_steps)}\n"
        md += f"- **Known biases:** {'; '.join(self.known_biases)}\n\n"
        
        md += f"## Evaluation Data\n"
        md += f"- **Source:** {self.eval_data_source}\n"
        md += f"- **Size:** {self.eval_data_size:,} samples\n\n"
        
        md += f"## Metrics\n"
        for metric, value in self.metrics.items():
            md += f"- **{metric}:** {value}\n"
        md += "\n"
        
        md += f"## Subgroup Performance\n"
        md += f"| Group | N | Accuracy | Precision | Recall | FPR | FNR |\n"
        md += f"|-------|---|----------|-----------|--------|-----|-----|\n"
        for row in self.subgroup_performance:
            md += f"| {row['group']} | {row['n']} | {row['accuracy']:.3f} | {row['precision']:.3f} | {row['recall']:.3f} | {row['false_positive_rate']:.3f} | {row['false_negative_rate']:.3f} |\n"
        md += "\n"
        
        md += f"## Ethical Considerations\n"
        for item in self.ethical_considerations:
            md += f"- {item}\n"
        md += "\n"
        
        md += f"## Limitations\n"
        for item in self.limitations:
            md += f"- {item}\n"
        md += "\n"
        
        md += f"## Recommendations\n"
        for item in self.recommendations:
            md += f"- {item}\n"
        
        return md
    
    def to_json(self) -> str:
        return json.dumps(asdict(self), indent=2)

Example: A Real Model Card

card = ModelCard(
    model_name="CustomerSupport-Sentiment-v2",
    model_type="Fine-tuned DistilBERT",
    version="2.1.0",
    date_trained="2026-07-15",
    authors=["ML Platform Team"],
    license="Proprietary",
    
    intended_uses=[
        "Classify customer support ticket sentiment as positive, neutral, or negative",
        "Prioritize negative-sentiment tickets for faster response"
    ],
    intended_users=["Customer support operations team"],
    out_of_scope_uses=[
        "Analyzing sentiment of non-English text",
        "Making automated decisions about customer account status",
        "Analyzing employee communications"
    ],
    
    training_data_source="Internal support tickets, 2024-2025",
    training_data_size=150000,
    training_data_demographics={
        "languages": {"English": 92, "Spanish": 5, "Other": 3},
        "regions": {"US": 60, "EU": 25, "APAC": 15}
    },
    preprocessing_steps=[
        "Removed PII (names, emails, phone numbers)",
        "Normalized whitespace and encoding",
        "Truncated to 512 tokens"
    ],
    known_biases=[
        "Training data is 92% English — non-English performance is unreliable",
        "Tickets from APAC region are underrepresented (15% of data)",
        "Sarcasm detection is weak — sarcastic negative tickets may be classified as neutral"
    ],
    
    eval_data_source="Held-out 10% of tickets, stratified by sentiment label",
    eval_data_size=15000,
    metrics={"accuracy": 0.91, "macro_f1": 0.87, "negative_recall": 0.94},
    subgroup_performance=[
        {"group": "English tickets", "n": 13800, "accuracy": 0.92, "precision": 0.91, "recall": 0.90, "false_positive_rate": 0.04, "false_negative_rate": 0.06},
        {"group": "Spanish tickets", "n": 750, "accuracy": 0.71, "precision": 0.68, "recall": 0.65, "false_positive_rate": 0.15, "false_negative_rate": 0.20},
        {"group": "Other languages", "n": 450, "accuracy": 0.58, "precision": 0.52, "recall": 0.48, "false_positive_rate": 0.22, "false_negative_rate": 0.30},
        {"group": "Overall", "n": 15000, "accuracy": 0.91, "precision": 0.89, "recall": 0.88, "false_positive_rate": 0.05, "false_negative_rate": 0.08},
    ],
    
    ethical_considerations=[
        "Sentiment classification should not be the sole basis for ticket prioritization — a neutral-sentiment ticket may still be urgent",
        "Non-English tickets have significantly lower accuracy (58-71%) — using this model to prioritize non-English tickets may systematically deprioritize non-English-speaking customers",
        "Sarcasm misclassification may cause genuinely upset customers to receive slower response"
    ],
    limitations=[
        "Only supports English reliably (92% accuracy vs 58% for other languages)",
        "Cannot detect sarcasm or irony",
        "Performance degrades on very short tickets (< 10 words)",
        "Has not been evaluated on tickets from before 2024"
    ],
    recommendations=[
        "Do not use for non-English ticket prioritization without additional language-specific models",
        "Monitor accuracy by language in production — alert if non-English accuracy drops below 60%",
        "Pair with a keyword-based urgency detector for short tickets",
        "Retrain quarterly with recent ticket data to handle drift"
    ]
)

# Generate the model card
with open("MODEL_CARD.md", "w") as f:
    f.write(card.to_markdown())

You're reviewing the model card above for the sentiment analysis model. The subgroup performance shows 92% accuracy for English tickets but 58% for other languages. A PM wants to deploy it for all tickets regardless of language. What's the correct action based on the model card?

The model card explicitly documents this limitation in three sections: out-of-scope uses ('Analyzing sentiment of non-English text'), subgroup performance (58% accuracy), and ethical considerations ('may systematically deprioritize non-English-speaking customers'). The correct action is to NOT deploy for non-English tickets and instead invest in language-specific models. Deploying a model with 58% accuracy for 3-class classification (where random is 33%) means nearly half of non-English tickets would be misclassified — this creates real harm for non-English-speaking customers.

Model Cards and Compliance

The EU AI Act (effective 2026) requires documentation similar to model cards for high-risk AI systems. Key requirements:

Description of the system's intended purpose
Accuracy, robustness, and cybersecurity metrics
Data quality and bias mitigation measures
Human oversight measures
Description of training, validation, and test data

If you're building models for EU users, model cards aren't optional — they're legally required documentation.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Build, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card.