How to Run an ML Fairness Audit: A Practical Checklist

Contents

Figure 1: How to Run an ML Fairness Audit: A Practical Checklist — nine phases, from scoping the decision to scheduling the next audit

Somebody asks, "Is our model fair?" and the room goes quiet. Nobody knows which metric to compute, which groups to check, or what number counts as fine.

A fairness audit answers that question with evidence instead of vibes. It is a structured review of what your model predicts, who it affects, how it performs for different groups, and what you will do about the gaps.

This guide gives you a phase-by-phase checklist you can run on a real system, with code for the measurement steps. I wrote this from my knowledge of standard fairness practice and common tooling, with no new research behind it, so verify the legal references and library details before you rely on them. This is not legal advice, and your counsel should weigh in on anything with regulatory exposure.

What a Fairness Audit Is (and Isn't)

A fairness audit checks whether a system produces unjustified disparities across groups, and it documents what you found and what you did about it.

It is not a certificate. No single audit proves a model is fair, because fairness has multiple competing definitions, and your system, data and users keep changing. Think of an audit as a snapshot with a repeatable method, which is why the report matters as much as the numbers.

Phase	Question it answers	Output
0. Scope and governance	What are we auditing, and who owns it?	Named owner, reviewer, timeline
1. Understand the system	What does it decide, and with what inputs?	Documented decision, label, features
2. Audit the data	Where could bias enter before the model?	Representation, label and proxy findings
3. Choose metrics	Which harm are we measuring?	Chosen metrics, thresholds, rationale
4. Measure	What are the actual gaps?	Per-group tables with intervals
5. Diagnose	Why do the gaps exist?	Error, feature and counterfactual analysis
6. Mitigate and retest	What fixes them, and what breaks?	Before-and-after results
7. Document	What is the evidence and the residual risk?	Signed audit report
8. Monitor	How will we know when it drifts?	Alerts, schedule, appeal loop

Phase 0: Scope and Governance

Settle these before you touch any data:

1. Pick the system and the decision. Which model, which decision, which population?
2. Name an audit owner with authority to recommend pausing or changing the system.
3. Seek independence. The people who built the model should not be the only people who judge it. An internal reviewer from another team or an outside auditor adds credibility.
4. Involve legal and compliance early. Some jurisdictions require specific audits for certain uses, such as hiring tools. As one example, New York City's Local Law 144 requires bias audits of automated employment decision tools, with a specific impact-ratio method. Check current requirements for your location and use case.
5. Decide how you will handle sensitive data during the audit: access controls, retention and purpose limits.
6. Set a timeline and a decision process for what happens if the audit finds problems.

Step 6 is the one teams skip, and it is the one that makes the audit matter. An audit with no pre-agreed consequence is a research project, not a control.

Phase 1: Understand the System

You cannot audit what you do not understand. Document these items:

• The decision and its stakes. Does the model approve loans, rank candidates, flag patients or suggest content? What happens to people on the wrong side of an error?
• The target label. What does the model actually predict? How does that relate to what you care about? The healthcare algorithm that predicted cost and missed illness is the canonical failure.
• The inputs. List every feature, including indirect ones like ZIP code, school or device type.
• The affected groups. Who experiences the decision, including people outside your customer base?
• Human involvement. Do people review, override or simply rubber-stamp the output?
• Existing documentation. Gather model cards, data sheets and past evaluations.

My view is that the label review pays off more than any other single step in this phase. Biased proxies cause the biggest, hardest-to-see problems, and they are invisible if you start at the metrics.

Phase 2: Audit the Data

Bias often enters before the model does. Check:

• Representation. How many examples exist for each group? Tiny groups produce noisy estimates and weak models.
• Label quality. Who produced the labels, and could the process itself be biased? Historical decisions, like past hiring or past arrests, carry past bias.
• Missing data. Does missingness differ by group?
• Measurement quality. Do sensors, forms or processes work equally well for everyone?
• Proxies. Which features correlate strongly with protected characteristics? Test by training a simple model to predict the protected attribute from your features. If it succeeds, proxies exist.
• Protected attribute availability. You cannot measure disparities without group labels. If you do not collect them, you may need self-reported data held separately for audit purposes, or carefully validated inference methods that carry their own error and ethical risks. Involve legal and privacy experts.

The proxy test is cheap and it is the highest-yield check in this phase. A gradient-boosted model that predicts group membership from your features at 0.85 AUC tells you the feature set carries the attribute, whatever the column names say.

Phase 3: Choose Fairness Metrics on Purpose

No metric works everywhere, and the common ones conflict. Link each metric to the harm you care about:

Metric	Zero gap means	Use it when
Selection rate	Demographic parity	Access to an opportunity is the harm, like being screened out of a job pool
True positive rate	Equal opportunity	Qualified people should be recognised at equal rates, like loan approvals for creditworthy applicants
False positive rate	Equalised odds, one half	Wrongly flagging people causes harm, like fraud flags or risk scores that trigger penalties
Precision	Predictive parity	A positive prediction should mean the same thing for each group
Calibration by group	Scores mean the same thing	Scores drive decisions, so a 0.7 should mean about 70% for every group

Write down which metrics you chose, why, and what you ruled out. Researchers have shown that when base rates differ you generally cannot satisfy all of them at once, so the choice reflects a value judgement that stakeholders should see. The pre-, in- and post-processing comparison walks through what happens to the other gaps when you optimise one.

Set Thresholds Before You Look

Decide in advance what counts as a problem. Some teams use the four-fifths rule from U.S. employment guidance as a screening heuristic: if one group's selection rate falls below 80% of the highest group's rate, flag it for review. Treat it as a rule of thumb, not a legal safe harbour, and check current guidance — the four-fifths rule explainer covers where it breaks. Pre-committing to thresholds prevents you from rationalising results afterwards.

Phase 4: Measure

Now compute the numbers. Libraries like Fairlearn, AIF360 and Aequitas handle much of the work. Here is a Fairlearn sketch:

from fairlearn.metrics import (
    MetricFrame, selection_rate, true_positive_rate, false_positive_rate,
)
from sklearn.metrics import accuracy_score

mf = MetricFrame(
    metrics={
        "accuracy": accuracy_score,
        "selection_rate": selection_rate,
        "tpr": true_positive_rate,
        "fpr": false_positive_rate,
    },
    y_true=y_true,
    y_pred=y_pred,
    sensitive_features=group,
)
print(mf.by_group)      # metrics per group
print(mf.difference())  # largest gap between groups
print(mf.ratio())       # smallest-to-largest ratio

difference() and ratio() are the two lines that turn a table into a finding, and the ratio is the quantity the four-fifths screen uses.

Add Uncertainty

A gap of three points means little if a group has 40 examples. Bootstrap the gap to get a confidence interval:

import numpy as np

def bootstrap_gap(y_pred, group, a, b, n=2000, seed=0):
    rng = np.random.default_rng(seed)
    idx_a = np.where(group == a)[0]
    idx_b = np.where(group == b)[0]
    gaps = [
        y_pred[rng.choice(idx_a, len(idx_a))].mean()
        - y_pred[rng.choice(idx_b, len(idx_b))].mean()
        for _ in range(n)
    ]
    return np.percentile(gaps, [2.5, 97.5])

Report intervals next to point estimates, and flag groups with too little data for a confident conclusion. An interval that spans zero is a finding too: it says the gap you were about to present is not distinguishable from noise at your sample size.

Slice Thoughtfully

• Single attributes: race, gender, age, disability and others relevant to your context.
• Intersections: check combinations, like older women or younger Black men. Disparities can hide in intersections even when single-attribute checks look clean.
• Other slices: region, language, device and data source.
• Thresholds and score ranges: check performance across the full range, not only at the operating point.

Intersections are where most audits stop one step early. Five attributes checked one at a time is five checks; the combinations are where the compounding harm lives.

Phase 5: Diagnose the Causes

A gap tells you something is wrong. It does not tell you why. Dig in:

• Error analysis: read examples of mistakes for each group. Patterns often jump out.
• Feature analysis: check which features drive predictions for different groups, using attribution methods with the usual caveats. Permutation importance and partial dependence are the fastest starting points.
• Counterfactual tests: change a protected attribute or a proxy in an otherwise identical record, and see whether the prediction moves.
• Data lineage: trace the label and feature pipeline for sources of bias.
• Threshold effects: check whether one threshold yields different error trade-offs across groups.

Phase 6: Mitigate and Retest

Options fall into three families:

Family	What you change	Main trade-off
Fix the data	Collect more representative data, relabel, reweight, or fix measurement problems	Cannot recover information that was never recorded
Change the model	Add fairness constraints during training, or change the target label (often the best fix)	Complexity, and a constraint tied to one model type
Adjust decisions	Tune thresholds, or add human review for specific cases	Group-specific adjustments can raise legal questions

Every mitigation carries trade-offs in accuracy, complexity, and sometimes legality. Some group-specific adjustments raise legal questions, so involve counsel. After you apply a fix, rerun the full audit, since a fix for one metric can worsen another — that is the pattern the mitigation comparison demonstrates with numbers.

Phase 7: Document Everything

An audit nobody can read is an audit that never happened. Your report should include:

1. Scope: system, decision, dates and version.
2. Data description: sources, sizes, group counts and known gaps.
3. Metrics and thresholds, with the reasons for each choice.
4. Results: tables with confidence intervals, plus intersectional findings.
5. Diagnosis: causes you found or suspect.
6. Mitigations: what you changed, and before-and-after results.
7. Residual risk: what remains, and who accepted it.
8. Limitations: what you could not test, such as missing group labels.
9. Sign-off: names, roles and dates.

Items 7 and 8 are what separate an audit from marketing. A report that lists no limitations has not looked hard enough, and one with no named risk acceptance has no owner.

Phase 8: Monitor and Re-Audit

Fairness drifts as data, users and the world change. Build a rhythm:

• Track key fairness metrics in production, with alerts on gaps that exceed your thresholds.
• Re-audit on a schedule, such as annually or after major retraining, data changes or product changes.
• Re-audit after incidents, complaints or regulatory changes.
• Collect feedback and appeals, and treat them as audit inputs.
• Track vendor changes if third-party models sit in your pipeline.

The monitoring programme in the AI risk assessment guide is where these triggers usually live, and it is the difference between an audit and a control.

Special Notes for LLM Systems

Generative systems need adapted methods, since there is no single prediction to score:

• Paired prompts: use prompts that differ only in a demographic detail, and compare quality, tone, helpfulness and refusals.
• Stereotype and toxicity probes: test outputs for stereotyping and harmful associations across groups.
• Language and dialect coverage: compare performance across languages and dialects.
• Downstream use: if an LLM screens, ranks or summarises people, run the full audit on that downstream task.
• Judge bias: if an LLM judge helps score outputs, calibrate it first.

That last point is the trap in 2026 audits. If your judge is biased, your fairness measurement is measuring the judge, and you will report a clean number with confidence.

Common Pitfalls

1. Skipping the label review. Biased proxies hide there, and no metric choice compensates for a target that is not the goal.
2. Choosing metrics after seeing results. Pre-commit to metrics and thresholds, or you are fitting the definition to the outcome.
3. Ignoring uncertainty. Small groups produce misleading gaps in both directions.
4. Checking only single attributes. Intersections hide disparities.
5. Auditing the model and ignoring the process. Human reviewers and workflows matter too, and a rubber stamp is part of the system.
6. Treating the audit as a one-time event. Systems drift, and so do thresholds.
7. Writing no report. Undocumented work cannot earn trust or survive a staff change.

Practical checklist:
SCOPE
[ ] System, decision and population defined
[ ] Audit owner and independent reviewer named
[ ] Legal and compliance consulted
UNDERSTAND
[ ] Target label and real goal compared
[ ] Inputs and proxies listed
[ ] Affected groups and human oversight documented
DATA
[ ] Group representation counted
[ ] Label and measurement quality reviewed
[ ] Proxy test run
MEASURE
[ ] Metrics chosen with written rationale
[ ] Thresholds set before analysis
[ ] Per-group and intersectional results with confidence intervals
ACT
[ ] Causes diagnosed
[ ] Mitigations applied and full audit rerun
[ ] Residual risk accepted by a named person
SUSTAIN
[ ] Report written and stored
[ ] Monitoring and alerts live
[ ] Next audit scheduled

Check the edition date before you buy. Library APIs move faster than print, so verify any code against the current Fairlearn and AIF360 documentation.

Unlock AI That Actually Works

Get lifetime access to GPT-6 Astra, Claude Fable 5.1, Gemini 3.5, Grok 4.5, and more — all in one platform. Build websites, apps, videos, content, and digital products from a single command. No monthly fees. No tool-hopping.

Click here to get GPTAstra Max now — one-time payment, lifetime access.

Frequently Asked Questions

What is an ML fairness audit?

It is a structured review of what a model predicts, who it affects, how it performs across groups, and what the organisation will do about the gaps it finds. The output is evidence: measured disparities, confidence intervals, diagnosed causes, mitigations tried and residual risk accepted by a named person. It is not a certificate. Because fairness has several competing definitions and your data and users keep changing, an audit is best treated as a snapshot taken with a repeatable method rather than a permanent verdict.

How long does a fairness audit take?

It depends on access to group labels and on how well the system is documented. The measurement phase is usually days, because the code is short, but scope, label review and diagnosis take longer, and the first audit of a system is dominated by finding out what the model actually predicts. A workable pattern is a two to four week first pass on one high-stakes model, then a lighter recurring audit once the pipeline, thresholds and report template exist. Re-audits after drift or retraining are much cheaper than the first one.

Which fairness metric should I choose?

The one that matches the harm. Selection rate suits access-to-opportunity harms such as being screened out of a job pool. True positive rate suits cases where qualified people should be recognised equally, such as creditworthy applicants. False positive rate suits cases where wrongly flagging someone causes the harm, such as fraud flags. Precision suits cases where a positive prediction should mean the same thing across groups. Calibration suits score-driven decisions. When base rates differ you generally cannot satisfy all of them, so write down the choice and what you ruled out.

Do I need protected-attribute data to audit fairness?

You cannot measure group disparities without group labels, so yes, in some form. The usual approach is voluntary self-identification, stored separately from the decision pipeline and used only for auditing, with access controls and retention limits. Inference from names or addresses carries its own error rates and legal and ethical risk, so treat it as a last resort and disclose its limits. If you truly cannot obtain labels, say so in the report as a limitation rather than claiming the system was audited.

Can a fairness audit prove a model is fair?

No. No single audit proves a model is fair, because fairness has multiple competing definitions and the system, its data and its users keep changing. What an audit can do is show that specific disparities were measured at specific thresholds, that known harms were mapped to chosen metrics, that causes were investigated, and that someone with authority accepted the residual risk. That is a defensible position; a claim of proven fairness is not.

How do you audit an LLM for fairness?

Adapt the method, because there is no single prediction to score. Use paired prompts that differ only in a demographic detail and compare quality, tone, helpfulness and refusals. Run stereotype and toxicity probes across groups. Compare coverage across languages and dialects. If the model screens, ranks or summarises people, audit that downstream task with the full process. And if an LLM judge helps score outputs, calibrate the judge first, since its own biases will otherwise be measured as the system's fairness.

Sources

• Fairlearn: fairness assessment user guide — MetricFrame and the metric definitions used above.
• Fairlearn: MetricFrame API — by_group, difference() and ratio().
• AI Fairness 360 documentation — metrics, datasets and mitigation algorithms.
• Aequitas — the audit toolkit from the Center for Data Science and Public Policy.
• NIST AI Risk Management Framework — the governance structure this checklist fits inside.
• NYC Department of Consumer and Worker Protection: automated employment decision tools — Local Law 144 guidance. This page blocks bot requests, so open it in a browser.

Links checked on 21 October 2026. Library APIs change; verify any snippet against the current docs before you run it in production.

Wrapping Up

A fairness audit turns a vague worry into a repeatable process: scope the system, question the label, audit the data, choose metrics on purpose, measure with uncertainty, diagnose, fix and document. Then keep watching, because fairness is not a state you reach once.

Start with your highest-stakes model and one honest question: what does this model predict, and does that match what we care about? That question alone has caught some of the most serious bias problems on record.

Which model in your estate would you least like to explain to a regulator, and when is its audit scheduled?

Related Articles

• Disparate Impact in ML: The Four-Fifths Rule Explained
• Bias Mitigation Techniques: Pre-, In-, and Post-Processing Compared
• Algorithmic Bias Examples in Hiring, Lending and Healthcare
• How to Run an AI Risk Assessment: Template and Process
• Permutation Feature Importance Explained with Scikit-learn