Phase 8: Evaluation, Safety & Observability · 50 min · Hugging Face Model Cards · MLflow · Jinja2

Model Cards: Documenting ML Models for Transparency and Compliance

If you can't document it, you don't understand it. If you don't document it, nobody trusts it.

Hiring signal: Model cards are increasingly required by regulation (EU AI Act) and internal governance. Engineers who can produce proper model documentation are qualified for ML Engineer roles at companies with mature ML practices.

What you will learn

Understand what model cards are and why they're now legally required in some jurisdictions
Write a complete model card following the Mitchell et al. framework
Document model performance across demographic subgroups
Identify ethical considerations, limitations, and recommended use cases
Generate model cards programmatically from training metadata

The Problem

A hospital deploys a model that predicts patient readmission risk. Doctors use it to decide which patients get extra follow-up care. Six months later, a study finds the model systematically underestimates risk for Black patients — they're denied follow-up care at higher rates than white patients with the same clinical profile.

The hospital didn't know because:

No one documented what population the model was trained on
No one tested performance across demographic subgroups
No one wrote down the model's intended use cases or limitations
No one created a model card

This actually happened. The 2019 Obermeyer et al. study found a widely-used healthcare algorithm discriminated against Black patients, affecting 200M people. The algorithm had no model card, no subgroup analysis, no documented limitations.

The Concept

What Is a Model Card?

A model card is a structured documentation file that accompanies a machine learning model. It answers:

What does this model do? (intended use, task, output)
How was it trained? (data, algorithm, training process)
How well does it work? (metrics, subgroup performance, eval conditions)
When should you NOT use it? (limitations, out-of-scope uses, ethical risks)
Who is responsible? (authors, contact, maintenance)

Model cards were proposed by Mitchell et al. (2019) and have become the industry standard, adopted by Hugging Face, Google, TensorFlow Hub, and required by the EU AI Act for high-risk AI systems.

The Model Card Template

# Model Card: [Model Name]

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Model Details, Intended Use, Training Data, Evaluation Data, Metrics, Subgroup Performance, Ethical Considerations, Limitations, Recommendations, Build It, Ship It, Model Details, Intended Use, Training Data, Evaluation Data, Metrics, Subgroup Performance, Ethical Considerations, Limitations, Recommendations, Use It, Common Pitfalls, Sources, Evaluation, Exercises, Key Terms, Interview Framing — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.