Phase 5: Evaluation & Quality for AI Products · 45 min · Hallucination rate · Demographic parity · Equalized odds
Measuring Hallucination, Bias & Safety
You will never compute a fairness metric yourself. You will always be the person deciding whether the number that comes back is acceptable.
Hiring signal: Responsible AI is now core PM scope, not a compliance sidebar, and interviewers probing 'how you ship responsibly under uncertainty' expect a candidate to interpret a hallucination rate, a demographic parity gap, or a toxicity score correctly on the spot -- including knowing what each metric can't tell you. A PM who nods along to any number that looks small enough is a liability; the hiring bar is a PM who asks the right follow-up question.
What you will learn
- Explain what a hallucination rate metric measures and its most common blind spots
- Interpret demographic parity and equalized odds at a level sufficient to question a data science team's fairness claim
- Explain what a toxicity/safety score covers and does not cover
- Ask the right follow-up questions when handed a metric result, without needing to compute the metric personally
The Problem
A data science lead hands a PM a slide before a launch review: "hallucination rate: 3.2%. Demographic parity difference: 0.04. Toxicity: 0.1% flagged." The PM has thirty seconds to decide whether this is good news. Nodding along because the numbers look small is how a PM ends up defending a launch decision they didn't actually understand, in the postmortem, after something the numbers didn't measure goes wrong. The job here isn't to compute these metrics -- that's data science and ML engineering's work, correctly. The job is to know what each metric measures, what it structurally cannot catch, and which follow-up question turns a slide into an actual decision.
This lesson covers the three metric families that show up most often in a launch review for a generative AI feature: hallucination, bias/fairness, and safety/toxicity.
Hallucination Rate
Hallucination rate measures the fraction of outputs containing a claim not supported by the provided source material or ground truth (this is the same underlying concept as the "faithfulness" metric from Lesson 3, usually reported as an error rate rather than a pass score). A 3.2% hallucination rate means roughly 1 in 31 outputs contained an unsupported claim, on whatever evaluation set produced that number.
The follow-up questions that turn this number into a decision:
- What set was this measured on? A hallucination rate from the golden set (Lesson 1) reflects known categories; a rate from shadow testing (Lesson 4) reflects real traffic distribution, which is usually the more trustworthy number for a launch call.
- Is the error uniform, or concentrated? A 3.2% overall rate that's actually 0.5% on routine queries and 18% on a specific query type (say, anything involving dates or numbers) is a very different risk profile than a flat 3.2% -- the aggregate number can hide exactly the category that matters most.
- What's the severity distribution, not just the rate? A hallucinated typo in a casual chat feature and a hallucinated dosage in a medical-information feature are both "hallucinations" at the same rate, but not remotely the same risk. The rate alone never tells you this -- you need the category breakdown from Lesson 1's golden set structure.
Unlock the full lesson
You've read the first 2 sections. The rest of this lesson covers Bias & Fairness Metrics, Safety & Toxicity Scoring, Interpreting Metrics You Didn't Compute, Build It, What to Practice — plus a hands-on lab, quiz, and project artifact.
Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.
Browse all courses · View pricing · DeVenture Academy