Equalized Odds (Fairness)
1. Definition
- Equalized Odds = a fairness criterion requiring that both true positive rates (TPR) and false positive rates (FPR) are the same across demographic groups.
Formally:
$P(\hat{Y}=1 \mid Y=y, A=a) = P(\hat{Y}=1 \mid Y=y, A=b) \quad \forall y \in \{0,1\}, \, \forall a,b$
Where:
- $\hat{Y}$ = model prediction
- $Y$ = true label
- $A$ = protected attribute (e.g., gender, race)
This means:
- Among those who should be positive ($Y=1$), each group has the same chance of being predicted positive (equal TPR).
- Among those who should be negative ($Y=0$), each group has the same chance of being incorrectly predicted positive (equal FPR).
2. Example
Loan approval model:
- Group A (men):
- TPR = 80% (qualified applicants approved)
- FPR = 20% (unqualified applicants wrongly approved)
- Group B (women):
- TPR = 60%
- FPR = 30%
This violates Equalized Odds, because both TPR and FPR differ across groups.
3. Why It’s Important
- Stronger fairness notion than Equal Opportunity.
- Ensures that errors (both false negatives and false positives) are not disproportionately distributed across demographic groups.
- Relevant in high-stakes domains like healthcare (misdiagnosis), criminal justice (recidivism prediction), lending.
4. Relation to Other Fairness Metrics
- Demographic Parity → requires equal overall positive prediction rates across groups (ignores labels).
- Equal Opportunity → requires equal TPR only.
- Equalized Odds → requires equal TPR and FPR.
Hierarchy:
- Demographic Parity (weakest, ignores ground truth)
- Equal Opportunity (stronger, considers TPR fairness)
- Equalized Odds (strongest, considers both TPR & FPR).
5. Limitations
- Harder to satisfy in practice, especially when base rates differ across groups.
- Can reduce overall accuracy (trade-off between fairness and performance).
- Sometimes too strict for real-world applications (may require relaxing to “approximate Equalized Odds”).
Summary:
Equalized Odds = equal true positive and false positive rates across groups.
It ensures fairness in both correct recognition and errors, but is stricter and harder to achieve than Demographic Parity or Equal Opportunity.