Fairness Guardrails

Fairness guardrails are the operational form of a fairness criterion: a metric, a set of groups, a threshold, and an action when the threshold is crossed. “Flag for review if the recall gap between evaluated subgroups exceeds 3 percentage points” is a guardrail. The criterion is the idea; the guardrail is what runs in the pipeline.

They exist because aggregate metrics hide subgroup behavior. A lending model can hold 90% overall accuracy while recovering 90% of qualified men and 60% of qualified women, and no global number will say so.

Passing a guardrail does not establish that a model is fair, and a gap does not establish unlawful discrimination. What a guardrail does is force a look. The difficulty starts when a governance document specifies several of them.

A threshold sweep with no passing point

import numpy as np

rng = np.random.default_rng(0)
n = 400_000
group = rng.integers(0, 2, size=n)
score = np.where(group == 0, rng.beta(2, 5, size=n), rng.beta(3, 3, size=n))
y = (rng.random(n) < score).astype(int)
"""One calibrated score, two groups, base rates 0.2844 and 0.5000. Group 1 is
   held at a threshold of 0.500; group 0's threshold is what we are choosing."""

def rates(g, t):
    m, flag = group == g, score >= t
    tp, fp = int((flag & m & (y == 1)).sum()), int((flag & m & (y == 0)).sum())
    fn = int((~flag & m & (y == 1)).sum())
    return (tp + fp) / m.sum(), tp / (tp + fp), tp / (tp + fn)

ref_sel, ref_ppv, ref_tpr = rates(1, 0.5)
print(f"{'group 0 threshold':>18}  {'selection ratio':>15} {'precision gap':>14} {'recall gap':>11}   guardrails passed")
for t in (0.20, 0.26, 0.30, 0.40, 0.50, 0.58, 0.70):
    sel, ppv, tpr = rates(0, t)
    checks = [sel / ref_sel >= 0.95, abs(ppv - ref_ppv) <= 0.03, abs(tpr - ref_tpr) <= 0.03]
    passed = [nm for nm, ok in zip(["approval rate", "precision", "recall"], checks) if ok]
    print(f"{t:>18.2f}  {sel / ref_sel:>15.4f} {ppv - ref_ppv:>+14.4f} {tpr - ref_tpr:>+11.4f}"
          f"   {', '.join(passed) if passed else 'none'}")
print("\nnone of the tested thresholds passes all three; no tested point passes both approval and precision")
#  group 0 threshold  selection ratio  precision gap  recall gap   guardrails passed
#               0.20           1.3078        -0.2867     +0.1938   approval rate
#               0.26           1.0175        -0.2468     +0.0764   approval rate
#               0.30           0.8379        -0.2190     -0.0121   recall
#               0.40           0.4646        -0.1416     -0.2360   none
#               0.50           0.2165        -0.0641     -0.4309   none
#               0.58           0.1012        -0.0048     -0.5404   precision
#               0.70           0.0222        +0.1053     -0.6265   none
#
# none of the tested thresholds passes all three; no tested point passes both approval and precision

Three guardrails, each of which would look uncontroversial in a model card. An approval-rate ratio of at least 0.95, echoing the four-fifths rule with a stricter bar. A precision gap within 3 percentage points. A recall gap within 3 percentage points. The only lever is group 0’s threshold.

None of the seven tested thresholds passes all three guardrails for this score with group 1 fixed at 0.5. At 0.26 the approval-rate guardrail passes and precision is 24.68 points apart. At 0.30 recall passes and the approval ratio is 0.8379. At 0.58 precision passes and group 0 is selected at about a tenth of group 1’s rate. An exhaustive threshold check or a proof is needed to extend a finite grid result to every threshold.

The identity relating prevalence, precision and error rates helps test feasibility, but it does not say that every set of approximate guardrails is impossible. For example, with prevalences 0.28 and 0.50, selecting 1% of each population entirely from its true positives gives equal selection rates, PPV 1 in both groups, and recall values 0.0357 and 0.0200. All three tolerances above pass. Recall is poor, so this is a reason to add a meaningful utility requirement, not a desirable policy. The model, reference operating point, required utility and allowed tolerances all matter.

Specify feasibility and conflict handling

Before choosing a priority order, check whether the complete requirements can be met in the intended operating range. Report which settings were searched and which constraints failed together. A trade-off should be a documented decision about harms and utility, not an assumption inferred from one unsuccessful threshold sweep.

• State the requirements and their reasons. Identify the harms, groups, metrics, minimum useful performance and tolerated gaps. Some requirements may be mandatory and cannot simply be traded away.
• Define what happens if the requirements conflict. Escalate for an explicit decision, change the model or workflow, or withhold deployment. Keep other harms visible; do not automatically demote their guardrails because a different metric was chosen as the primary one.
• Record the expected gap on the others. If recall parity is a requirement, measure the accompanying precision and selection-rate gaps for the proposed policy. Recall parity alone does not fix their values.
• Check whether the base rates differ at all. Equal base rates remove one source of arithmetic incompatibility; the available scores and policies still determine feasibility. Different base rates likewise do not exclude every simultaneous criterion.

Building the rest of the guardrail

• The groups. Protected attributes as the application and the law define them, plus fairness-relevant ones. Geography is worth evaluating and is not automatically protected — but it frequently proxies for something that is.
• The tolerance, set in advance and justified. Three percentage points is an operational choice, not evidence that three points are immaterial to the people inside them.
• Sample sizes and intervals. A subgroup of two hundred produces a recall estimate that moves several points on noise, so a guardrail without an interval will fire and clear at random.
• Continuous monitoring. Subgroup performance degrades with drift even when the model is unchanged, so a guardrail checked once at release is a snapshot.
• An investigation step before any action. A gap can come from data quality, label bias, sampling, the model, or the decision policy wrapped around it, and the remedies differ completely.

On remedies: pre-processing addresses representation and labels, in-processing adds constraints during training, post-processing adjusts outputs. Group-specific thresholds are the most direct post-processing lever and the most legally exposed, since they use a protected characteristic explicitly — not something to adopt without counsel in a high-stakes setting.

What guardrails are not

They are not an explanation. A model can produce an entirely innocuous feature-importance report — no protected attribute anywhere in it — while approving one group at three times the rate of another through a postcode variable, and the post-hoc explanation will be accurate the whole time. Only measuring outcomes by group finds that.

And they are not compliance. The four-fifths rule in 29 CFR 1607.4(D) yields evidence of adverse impact rather than a finding, and its own text says smaller differences may still constitute adverse impact. The GDPR requires fair processing without prescribing a statistical metric. A passing dashboard is an input to a legal assessment, not a substitute for one.

References

• Chouldechova, Fair prediction with disparate impact, 2017
• Kleinberg, Mullainathan & Raghavan, Inherent Trade-Offs in the Fair Determination of Risk Scores, ITCS 2017
• 29 CFR 1607.4, Uniform Guidelines on Employee Selection Procedures — Information on impact