Phase 2: Bias Detection & Fairness Metrics · 45 min · Python · pandas · scikit-learn
The Problem
In Lesson 1, you identified where bias comes from. Now you need to measure it. But here's the hard part: there is no single "fairness score." There are at least 4 metrics that measure different things, and they can disagree. A model that looks fair by one metric can look biased by another — and both can be mathematically correct.
This is what made the COMPAS debate so contentious. ProPublica said COMPAS was biased because it had unequal false positive rates across races (an equalized odds problem). Northpointe said COMPAS was fair because it had equal positive predictive values across races (a predictive parity argument). Both were right. The disagreement wasn't about the math — it was about which definition of fairness matters for a criminal justice context.
The 4 Core Fairness Metrics
1. Demographic Parity (Statistical Parity)
Definition: The model's positive prediction rate should be equal across groups.
Formula: P(ŷ = 1 | A = group_a) = P(ŷ = 1 | A = group_b)
When it matters: When the base rate of the outcome should be equal across groups — e.g., hiring, where you believe qualified candidates exist at equal rates across groups.
When it fails: When base rates legitimately differ — e.g., a medical model predicting pregnancy should not have demographic parity across sexes.
2. Equalized Odds
Definition: The model's true positive rate AND false positive rate should be equal across groups.
Formula: TPR(A = a) = TPR(A = b) AND FPR(A = a) = FPR(A = b)
When it matters: When the model's errors should be distributed equally across groups — e.g., criminal justice, where a false positive means someone is wrongly denied freedom.
3. Equal Opportunity
Definition: A relaxed version of equalized odds — only the true positive rate needs to be equal.
Formula: TPR(A = a) = TPR(A = b)
When it matters: When you care about equal access to positive outcomes but are less concerned about false positives — e.g., loan approval where you want equal opportunity to get a loan.
4. Disparate Impact Ratio (The 80% Rule)
Definition: The ratio of the positive prediction rate for the disadvantaged group to the advantaged group should be ≥ 0.80.
Formula: P(ŷ = 1 | A = disadvantaged) / P(ŷ = 1 | A = advantaged) ≥ 0.80
When it matters: This is the legal standard in the U.S. — the EEOC's "80% rule" used in employment discrimination cases. If your model's selection rate for a protected group is less than 80% of the rate for the advantaged group, it creates a presumption of disparate impact.
The 80% rule is law, not theory
The EEOC's 80% rule (formally the "four-fifths rule") is codified in the Uniform Guidelines on Employee Selection Procedures (29 CFR § 1607.4(D)). It's used in employment discrimination cases. If your hiring model selects women at a rate less than 80% of the rate for men, you have a prima facie case of disparate impact. This is not an academic metric — it's a legal standard.