Bias Mitigation Techniques: Pre-, In-, and Post-Processing

Contents

Figure 1: Bias Mitigation Techniques: Pre-, In-, and Post-Processing — three families, five gaps, and the metric that quietly doubles while you are watching the other one

Your model approves 46% of one group and 42% of another. You add a fairness fix, and the gap shrinks. Then someone checks the false-positive rate and finds it doubled. Did the fix work?

That is the real story of bias mitigation: every fix moves something else. Here are the three families of techniques, what they need, and what happened when I implemented one of each on data where I control the source of the bias. I am not a lawyer, so check legal questions with one.

The Three Families

Toolkits sort mitigation methods by where in the pipeline they act. The AIF360 docs give a clean list:

Family	Acts on	Examples from the AIF360 docs
Pre-processing	The training data	Reweighing, which weights the examples in each group and label combination differently; Disparate Impact Remover, which edits feature values while preserving rank-ordering within groups
In-processing	The learning step	Adversarial Debiasing; Prejudice Remover, which adds a discrimination-aware regularisation term to the learning objective
Post-processing	The model's outputs	Reject Option Classification; Equalized Odds Postprocessing, which solves a linear program to pick the probabilities for changing output labels

Fairlearn offers one of each too. Its page lists CorrelationRemover for preprocessing, ExponentiatedGradient as a reduction, and ThresholdOptimizer for postprocessing, which derives a monotone transformation of the classifier's prediction to enforce the specified parity constraints. The reductions approach works by turning fair classification into a sequence of cost-sensitive classification problems, which is why it can wrap almost any base learner.

For post-processing, Hardt and colleagues' abstract says you can optimally adjust any learned predictor to remove discrimination according to their definition, as long as you have data on the predictor, the target and group membership. They add that the framework shifts the cost of poor classification from disadvantaged groups to the decision maker, which is a useful sentence to bring to a product review.

What We're Measuring

I tracked five numbers, each as the gap between two groups, A and B:

Metric	What it measures	Parity at zero means
Selection rate	The share receiving a positive decision	Demographic parity
True-positive rate (TPR)	Among people who deserve a yes, the share who get one	Equal opportunity
False-positive rate (FPR)	Among people who deserve a no, the share who get a yes	Equalised odds, one half
Precision (PPV)	Among people who get a yes, the share who deserved it	Predictive parity
Accuracy	Overall correctness	Nothing about fairness on its own

Reporting only the gap you targeted is how the false-positive surprise happens. Five numbers is still cheap; one is a story you tell yourself.

My Test Setup

Fairlearn and aif360 were not installed and pip was unreachable, so I wrote one method from each family myself:

• Pre: reweighing, with weight equal to P(group) times P(label) divided by P(group, label).
• In: logistic regression with a penalty on the squared gap in average predicted probability between groups.
• Post: a separate decision threshold per group, fitted on a validation set.

The data is synthetic. Group B is 40% of rows, has a lower true base rate, and gets noisier measurements. A proxy feature correlates with group, and the model never sees the group column itself. I checked each method against its own definition: reweighing equalises the positive rate across groups, my penalised loss's gradient matches finite differences, and with the penalty off my fit agrees with scikit-learn on 100% of held-out decisions.

import numpy as np
import pandas as pd

def reweighing_weights(y, group):
    """weight = P(group) * P(label) / P(group, label), the AIF360 formulation."""
    df = pd.DataFrame({"y": y, "g": group})
    p_g = df["g"].value_counts(normalize=True)
    p_y = df["y"].value_counts(normalize=True)
    p_gy = df.groupby(["g", "y"]).size() / len(df)
    return df.apply(
        lambda r: p_g[r["g"]] * p_y[r["y"]] / p_gy[(r["g"], r["y"])], axis=1
    )

w = reweighing_weights(y_train, group_train)
print(w.groupby([group_train, y_train]).mean())

That print is the test. If the group means do not come out equal, the weights are wrong, and a silent weighting bug looks exactly like a fairness fix that did nothing.

Scenario 1: Accurate Labels, Unequal Data

Averages over 5 datasets, 7,500 test rows each:

Method	Accuracy	Selection gap	TPR gap	FPR gap	PPV gap
No mitigation	0.727	0.042	0.047	0.033	0.117
Pre: reweighing	0.726	0.015	0.019	0.065	0.134
In: penalty 50	0.726	0.010	0.010	0.075	0.140
Post: equal selection rate	0.724	0.017	0.014	0.075	0.140
Post: equal TPR	0.725	0.018	0.014	0.071	0.137

Three things stand out:

1. The targeted gaps shrank by two-thirds or more. Selection and TPR gaps dropped from about 0.04 to between 0.01 and 0.02.
2. Accuracy barely moved. The drop is at most 0.003, and accuracy varied by 0.003 to 0.004 between datasets, so I cannot distinguish the methods on accuracy.
3. Other gaps got worse. FPR gaps roughly doubled, from 0.033 to between 0.065 and 0.075, and PPV gaps grew from 0.117 to between 0.134 and 0.140.

The three families landed close together because I set similar targets, so do not read this as a ranking. The real differences are in what each needs, which the table further down covers.

Scenario 2: Biased Historical Labels

Now suppose 30% of group B's true positives were recorded as negatives. The models train on the biased labels. I scored them against the true labels they never saw:

Method	Accuracy (true)	TPR gap (true)	FPR gap (true)	PPV gap (true)
No mitigation	0.717	0.120	0.021	0.086
Pre: reweighing	0.719	0.018	0.057	0.135
In: penalty 50	0.715	0.024	0.070	0.144
Post: equal selection rate	0.713	0.021	0.061	0.138
Post: equal TPR	0.715	0.027	0.062	0.136

Against the truth, the TPR gap fell from 0.120 to about 0.02, with no clear accuracy cost, since accuracy varied by 0.006 to 0.008 across datasets. The FPR and PPV gaps grew again. In this scenario the mitigations helped because I built the bias in a form they counteract well. Real bias does not announce its type, which is the limitation of every result in this article.

Why One Fix Breaks Another

Kleinberg and colleagues formalise three fairness conditions and prove that, except in highly constrained special cases, no method can satisfy all three simultaneously. Their abstract also says that even satisfying all three approximately requires the data to lie in an approximate version of one of those cases. I read only the abstract and cannot confirm from it which three conditions they mean, so I am not mapping my five gaps onto them.

My tables show the practical face of that result. When groups' base rates differ, closing one gap tends to open another. Which gap matters is a value judgment, not a math problem, and the person who owns that judgment should be named in the risk register rather than implied by whoever last tuned the threshold.

How the Families Differ in Practice

This part is my judgment, based on how the methods work:

Dimension	Pre-processing	In-processing	Post-processing
Needs group labels	At training	At training	At training or validation, and at decision time in my version
Touches the model?	No	Yes, tied to a model type	No
Knob to tune	Few	Penalty strength	Target gap
Good when	You want any model downstream	You control the training code	You cannot retrain the model
Watch out for	Weights cannot fix missing information	Penalty tuning needs care	Using group at decision time may be restricted

On that last point, my post-processing rows use the group attribute when deciding. Whether that is permitted depends on the setting and the law, which I have not researched here. Hardt and colleagues note that their approach assumes group membership data is available, which tells you the assumption is explicit rather than incidental.

Common Pitfalls

1. Mitigating before measuring. Compute all five gaps on a held-out set first, or you will not know which story your fix tells.
2. Optimising the wrong gap. Wrongly denying qualified people points to TPR; wrongly approving points to FPR. Pick the one that matches the harm.
3. Comparing only the target. A method that halves the gap you named while doubling another is not obviously an improvement.
4. Trusting labels you have not checked. If your labels may be biased, find a cleaner sample and test against it, because a fix tuned on biased labels can look perfect and be wrong.
5. Stopping at launch. Distributions shift, and the gap you closed can reopen with a new population.
6. Treating a zero as fair. The toolkit's own documentation warns that mathematical guarantees on a metric do not make the results fair.
7. Shipping group-based thresholds without legal review. The technical fix can be legally unavailable even when it is statistically clean.

Before you call the mitigation done:
[ ] all five gaps measured before and after, on held-out data
[ ] the targeted gap named, and the harm it represents written down
[ ] the other four gaps reported alongside it, not hidden
[ ] accuracy compared against dataset-to-dataset variation
[ ] labels checked against a cleaner sample where bias is suspected
[ ] group-label requirements documented for whichever family you chose
[ ] legal review done if the method uses group membership at decision time
[ ] post-launch monitoring scheduled, with an owner

Check the edition date before you buy. The methods here are stable, but the toolkits move quickly, so check the current AIF360 and Fairlearn docs alongside any book.

Unlock AI That Actually Works

Get lifetime access to GPT-6 Astra, Claude Fable 5.1, Gemini 3.5, Grok 4.5, and more — all in one platform. Build websites, apps, videos, content, and digital products from a single command. No monthly fees. No tool-hopping.

Click here to get GPTAstra Max now — one-time payment, lifetime access.

Frequently Asked Questions

What is the difference between pre-processing, in-processing and post-processing bias mitigation?

They differ by where in the pipeline they act. Pre-processing edits the training data, for example reweighing examples so each group and label combination carries the same total weight, or adjusting feature values while preserving rank order within groups. In-processing changes the learning step, adding an adversarial component or a discrimination-aware penalty to the objective. Post-processing leaves the model alone and adjusts its outputs, usually by picking different decision thresholds or probabilities per group. Toolkits such as AI Fairness 360 and Fairlearn ship examples of all three families.

Do bias mitigation techniques reduce accuracy?

In my tests, barely. Across two scenarios the accuracy drop was at most 0.003 in the first and there was no clear cost in the second, while accuracy varied by 0.003 to 0.008 between datasets, which is larger than the differences between methods. That does not mean mitigation is free: the cost showed up in other fairness gaps, with the false-positive gap roughly doubling and the precision gap growing. Report accuracy next to every gap, and treat differences smaller than your dataset-to-dataset variation as ties.

Why does fixing one fairness metric break another?

Because the conditions conflict when groups have different base rates. Kleinberg and colleagues formalise three fairness conditions and prove that, except in highly constrained special cases, no method can satisfy all three at once, and even approximate satisfaction requires the data to sit near one of those cases. My tables show the practical version: closing the selection and true-positive gaps opened the false-positive and precision gaps. Which gap matters more is a value judgment about who bears the cost of each error, not a mathematical result.

Can pre-processing fix biased labels?

Partly, and only when the bias has a shape the reweighting can see. In my second scenario, where 30% of one group's true positives were recorded as negatives, reweighing cut the true-positive gap against the true labels from 0.120 to 0.018. But it cannot recover information that was never recorded, and it widened the false-positive and precision gaps. If your labels come from a biased process, the more useful step is finding a cleaner sample to validate against, rather than assuming a weighting scheme repairs the record.

Is post-processing allowed if it uses group membership at decision time?

It depends on the jurisdiction, the domain and the specific practice, and I have not researched the legal question here. Technically, the post-processing methods in my tests need the group attribute at decision time, and Hardt and colleagues note that their approach assumes group membership data is available. Some settings restrict using protected attributes in decisions even for remedial purposes. Check with counsel before shipping a per-group threshold, and consider whether a model-level method avoids the issue in your context.

Is a model with a zero fairness gap actually fair?

No. Fairlearn's own documentation makes the point: the algorithms provide mathematical guarantees about how close they can drive some unfairness metric to zero, but that does not mean the results are fair. A gap can close because the metric was the wrong one for the harm, because the labels were biased, or because the disparity moved into a measure you did not compute. Treat a zero gap as evidence about one number, not as a verdict, and keep the affected-community review alongside it.

Sources

• AI Fairness 360 — the toolkit paper, arXiv 1810.01943.
• AIF360 algorithms documentation — the pre-, in- and post-processing lists quoted above.
• Equality of Opportunity in Supervised Learning — Hardt, Price and Srebro, arXiv 1610.02413.
• A Reductions Approach to Fair Classification — Agarwal, Beygelzimer, Dudik, Langford and Wallach, arXiv 1803.02453.
• Inherent Trade-Offs in the Fair Determination of Risk Scores — Kleinberg, Mullainathan and Raghavan, arXiv 1609.05807.
• Fairlearn mitigation user guide — the three Fairlearn classes named in this article.

Links checked on 21 October 2026.

Wrapping Up

Pre-, in- and post-processing all moved the gaps they targeted, at little visible accuracy cost, and all three widened others. The families differ less in results than in requirements: what data they need, whether they touch the model, and when they need the group label.

The right order is measure, decide which harm matters most, mitigate, and re-measure everything. A fairness fix is not a number going down; it is a decision about who bears which error, made visible and then tested.

Which gap will you check first: the one you were about to fix, or the one your fix will quietly open?

Related Articles

• Algorithmic Bias Examples in Hiring, Lending and Healthcare
• AI Ethics Principles: Fairness, Accountability and Transparency
• How to Run an AI Risk Assessment: Template and Process
• Permutation Feature Importance Explained with Scikit-learn
• Interpretability vs Explainability in Machine Learning: What's the Difference?