Fairness Auditing in Practice: How to Actually Audit a Model — and Why Most Audits Are Theatre

Most writing about fairness auditing is either conceptual (“why do models discriminate?”) or a tool advertisement. The real gap is operational: what do you measure when you may not lawfully hold the protected attribute, which of the mutually contradictory metrics do you pick, how many examples does a subgroup need before the number means anything, and who is going to read the report? This post fills that gap — and explains why audits so often turn into theatre.

Do mandated audits work? The best evidence we have#

Mandating audits sounds sensible. What happens in practice? The most concrete answer available comes from New York City, whose rule on automated employment decision tools became enforceable on 5 July 2023, requiring employers to commission an annual independent bias audit and publish the results.

An analysis by Cornell researchers found, across roughly 400 employers, only 18 bias audits and 11 transparency notices (source ↗). The obligation exists; the compliance barely does.

An audit obligation that does not identify who must be audited, and attaches no cost to not auditing, degrades into a voluntary reporting regime. And in voluntary regimes only the already-good report.

The regulatory direction is not one-way either. Colorado repealed its state AI act and replaced it: the successor statute, signed 14 May 2026 and effective 1 January 2027, drops the impact assessment requirement and leaves enforcement solely with the attorney general (SB 26-189 ↗). The conclusion is simple: regulation will not make you audit. You audit because there is no other way to learn where your model breaks.

The first obstacle: you may not hold the protected attribute#

The most common well-founded refusal in this area is: “we do not hold ethnicity, religion or health data, so we cannot do subgroup analysis.” That sentence is made in good faith and is often legally correct — but it is not a complete answer.

Article 10 of the EU AI Act opens a narrow, tightly conditioned door to processing special categories of personal data specifically for the purpose of bias detection and correction. The conditions are strict and cumulative: that it cannot effectively be done otherwise, strict technical limitations and access controls, no transmission to third parties, deletion once the purpose is served, and documentation (Article 10 ↗).

What you can do without the attribute#

• Do not use a proxy variable — but do measure proxy reliance. Inferring ethnicity from postcode is both legally and statistically a bad idea. Measuring how heavily the model leans on postcode is legitimate and informative.
• Voluntary, segregated collection. Some organisations collect voluntary demographic data held apart from the product database and accessed only for audit.
• Aggregated external comparison. Without individual labels you can compare regional outcome rates against aggregated official statistics; this infers nothing about an individual but makes systematic skew visible.
• Coverage gaps instead of performance gaps. On which kinds of input does the model more often abstain or score near the threshold? That traces inequity without requiring a protected attribute at all.

Choosing a metric: you cannot satisfy them all#

Fairness metrics contradict each other, and that is a mathematical result rather than an implementation failure. When base rates differ across groups and the model is not perfect, the common criteria cannot all hold simultaneously. The conceptual background is in bias and fairness; here we focus on the selection rule.

• Demographic parity: the rate of positive decisions should be equal across groups. Reasonable in allocation problems (scholarships, ad delivery); nonsensical in diagnosis.
• Equalised odds / equality of opportunity: true positive rates (and where relevant false positive rates) should be equal. The right choice when error costs are asymmetric: in CV screening, a false negative means the candidate is never seen at all.
• Calibration parity: the same score should mean the same true probability in every group. Indispensable when a human will interpret the score; details in model calibration.

The selection rule fits in one sentence: look at which error harms whom, and how much. Choose the metric by the harm mechanism rather than by mathematical elegance, and write the choice down with its justification. An unwritten metric choice gets quietly revised when the result is unwelcome — which is precisely what turns an audit into theatre.

The honesty of an audit is measured by when the metric was chosen. A metric chosen before you look at the results is a commitment; one chosen afterwards is a defence.

Subgroup size: what you measured may be noise#

The most common technical error in audit reports is treating differences in small subgroups as real. When accuracy comes out at 72 % in a subgroup of 40, the uncertainty around that number is wide enough that the gap to the overall average could easily be chance.

The practical discipline:

• Give a confidence interval for every subgroup. An interval, not a point. For the method, see confidence intervals and bootstrap.
• Account for multiple comparisons. Examine twelve subgroups and you should expect at least one to look “significant” even when no real difference exists.
• Say so when the sample is too small. “We do not have enough data to draw a conclusion for this subgroup” is a legitimate and honest finding, and it beats a fabricated number.
• Watch the rare class. As a subgroup shrinks, its positive count shrinks faster and metrics become unstable; the mechanics are in class imbalance.

# Skeleton of an audit loop (pseudocode)
for group in protected_groups:
    n = len(test_set[group])
    if n < MIN_SAMPLES or positive_count(group) < MIN_POSITIVES:
        report(group, status="insufficient data")   # say it, do not invent
        continue
    metric   = chosen_criterion(model, test_set[group])
    interval = bootstrap_confidence_interval(metric, n_iter=2000)
    report(group, metric, interval, n)
# Compare on the DIFFERENCE between groups;
# eyeballing whether two separate intervals overlap is misleading.

Why audits become theatre#

No bad faith is required; the structure drifts there on its own. Four typical mechanisms:

• Scope narrowing. The audit covers where the data is cleanest rather than where the model is used most dangerously. The scope decision must be made before the audit and independently of the audited party.
• Metric shopping. The criterion is chosen after seeing the results. The antidote is to write and version the metric and threshold in advance.
• A snapshot. The audit happens once while the model keeps changing. Degradation in production is caught by the monitoring and drift discipline — and fairness metrics belong among the monitored metrics.
• A report nobody reads. With no action attached to a finding, an audit is a document-production exercise. Give every finding an owner and a date.

An audit protocol#

An ordered, repeatable and defensible protocol:

• 1. Write the scope. Which model version, which decision, which population, which time window.
• 2. Name the harm mechanism. Is it the false positive that harms, or the false negative? And whom?
• 3. Choose the metric and threshold in advance, with a justification, before you look at results.
• 4. Define the groups and the minimum sample threshold. List intersectional groups (age × gender, say) up front too; a group added afterwards is indistinguishable from searching.
• 5. Measure and report intervals. Report the difference and the interval around the difference.
• 6. Look for the cause. Does the gap come from under-representation, label noise, or feature choice? The data-centric approach is the most productive tool here.
• 7. Assign action and owner. Every finding needs a responsible person and a date.
• 8. Set the repeat date. An audit is a rhythm, not an event.

One closing caution: improving a fairness metric and making a model fairer are not the same thing. You can fix a metric by tuning a per-group threshold; whether that is legally and ethically acceptable is a separate question, and one engineering cannot answer alone. Which is why the first item in the protocol is scope, the last is repetition, and everything in between is written justification.

In short: done properly, a fairness audit is not a compliance artefact but a bug-finding tool. It tells you where, for whom, and by how much the model breaks. A team that wants that information does not fear the audit; for a team that does not, the audit was theatre from the start.