AI Ethics & Bias Mitigation

Fairness Metrics & Definitions

There is no universal definition of algorithmic fairness. Each formal fairness criterion captures a different moral intuition about what it means to treat people equally, and different criteria produce genuinely different verdicts on whether a given model is fair. The choice of fairness metric is therefore not a technical decision to be made by the data scientist alone — it is a values decision that must involve domain experts, legal counsel, and affected communities.

Individual vs. Group Fairness

Demographic parity (statistical parity) requires that the proportion of positive predictions is equal across demographic groups. If a loan approval model approves 60% of Group A applicants, it must also approve 60% of Group B applicants. Equal opportunity (Hardt et al., 2016) relaxes this: it requires equal true positive rates across groups — applicants who would genuinely repay a loan should be approved at the same rate regardless of group membership. Equalised odds extends equal opportunity to also require equal false positive rates, imposing parity on both error types simultaneously. Predictive parity (calibration within groups) requires that when the model predicts a 70% probability of default, 70% of such applicants actually default — regardless of group membership.

Fairness Definitions Comparison

The Impossibility Theorems

Alexandra Chouldechova's 2017 impossibility theorem formalised what the COMPAS debate had exposed empirically: when base rates differ across groups, it is mathematically impossible for a classifier to simultaneously satisfy calibration, equal false positive rates, and equal false negative rates. More precisely, if a model is calibrated and the prevalence of the outcome differs across groups, then the false positive rate and false negative rate must also differ across groups. This is not a limitation of any particular algorithm; it is an algebraic consequence of the relationship between base rates and error rates.

The practical implications for high-stakes applications are significant. In criminal justice, achieving calibration mathematically guarantees different false positive rates across groups when base rates differ — exactly the disparity ProPublica documented in COMPAS. In healthcare triage, prioritising equal opportunity will produce different positive predictive values across demographic groups. No technical intervention can dissolve these trade-offs; they are inherent to the problem structure. Practitioners must make explicit, documented choices about which criterion to prioritise for their specific use case, legal jurisdiction, and harm model.

Measuring Fairness in Code

The following Python function computes the core fairness metrics across demographic groups from model predictions, producing a structured audit report:

import pandas as pd
import numpy as np
from sklearn.metrics import confusion_matrix

def compute_fairness_metrics(y_true, y_pred, sensitive_attr, positive_label=1):
    """Compute key fairness metrics across demographic groups."""
    groups = pd.Series(sensitive_attr).unique()
    results = {}

    for group in groups:
        mask = np.array(sensitive_attr) == group
        y_true_g = np.array(y_true)[mask]
        y_pred_g = np.array(y_pred)[mask]

        tn, fp, fn, tp = confusion_matrix(y_true_g, y_pred_g, labels=[0, 1]).ravel()

        results[group] = {
            "n": mask.sum(),
            "selection_rate": (y_pred_g == positive_label).mean(),  # Demographic parity
            "true_positive_rate": tp / (tp + fn) if (tp + fn) > 0 else 0,  # Equal opportunity
            "false_positive_rate": fp / (fp + tn) if (fp + tn) > 0 else 0,  # Equal FPR
            "accuracy": (y_true_g == y_pred_g).mean(),
            "precision": tp / (tp + fp) if (tp + fp) > 0 else 0
        }

    df = pd.DataFrame(results).T

    # Fairness violations (disparate impact if ratio < 0.8)
    max_sr = df['selection_rate'].max()
    min_sr = df['selection_rate'].min()
    di_ratio = min_sr / max_sr if max_sr > 0 else 1.0

    print(f"\n=== FAIRNESS AUDIT REPORT ===")
    print(df[['n', 'selection_rate', 'true_positive_rate', 'accuracy']].round(3))
    print(f"\nDisparate Impact Ratio: {di_ratio:.3f} {'WARNING VIOLATION (<0.8)' if di_ratio < 0.8 else 'PASS'}")
    print(f"TPR Difference: {df['true_positive_rate'].max() - df['true_positive_rate'].min():.3f} {'WARNING VIOLATION (>0.1)' if df['true_positive_rate'].max() - df['true_positive_rate'].min() > 0.1 else 'PASS'}")

    return df

# Example: hiring algorithm audit
# Protected attributes: gender, race, age_group
# Found: selection_rate for women = 0.32 vs men = 0.48 → DI ratio = 0.67 → VIOLATION

The function surfaces two of the most commonly required metrics in practice: the disparate impact ratio (the 80% rule in US employment discrimination law requires a ratio no lower than 0.8) and the absolute difference in true positive rates between the best and worst performing demographic slices. Both should be tracked as first-class model evaluation metrics alongside AUC and F1.

Dataset Auditing & Documentation

Systematic dataset auditing must precede model development, not follow it. An undocumented dataset is a liability: it embeds unstated assumptions about who is represented, how labels were generated, what the collection context was, and what uses the data was originally consented for. When those assumptions are wrong — and they frequently are — they propagate silently through every model trained on the dataset, every evaluation run against it, and every deployment decision informed by it.

A structured audit covers four dimensions: provenance (where did the data come from, who collected it, under what conditions?); composition (what demographic subgroups are represented, at what frequencies, and how does that compare to the deployment population?); label quality (what was the annotation process, who were the annotators, what was inter-annotator agreement?); and legal and ethical review (was informed consent obtained, do any individuals have the right to be forgotten, are there jurisdiction-specific restrictions on using this data for the intended purpose?).

Datasheets for Datasets

Timnit Gebru and colleagues proposed "Datasheets for Datasets" (2018, Communications of the ACM 2021) as the dataset-level equivalent of electronics component datasheets. The framework organises documentation into seven sections: Motivation asks why the dataset was created and by whom; Composition documents what instances represent, how many there are, and known demographic gaps; Collection process records how data was gathered and whether consent was obtained; Preprocessing and labelling covers any transformations applied and inter-annotator agreement; Uses states what the dataset is and is not appropriate for; Distribution covers licensing and access conditions; Maintenance records who is responsible for updates and error correction.

Margaret Mitchell and colleagues proposed the complementary Model Card framework (2019) to extend documentation to trained models: each deployed model should be accompanied by a card that reports performance disaggregated across demographic groups, intended use cases, out-of-scope uses, and known limitations. The EU AI Act's transparency requirements for high-risk AI systems effectively mandate documentation comparable to these standards.

Auditing Tools & Code

Several open-source tools have matured into practical workhorses for fairness auditing. Aequitas (University of Chicago) focuses on classification audits, computing a full matrix of group fairness metrics. Fairlearn (Microsoft) combines metric computation with mitigation algorithms via the MetricFrame abstraction. AI Fairness 360 (IBM) offers the broadest coverage, with over 70 fairness metrics and more than 10 mitigation algorithms.

AI Bias Sources and Mitigations

The following function detects dataset bias through representation analysis, label rate disparities, and proxy discrimination via Cramér's V correlation:

import pandas as pd
from collections import Counter

def audit_dataset_bias(df: pd.DataFrame, sensitive_cols: list, target_col: str):
    """Detect representation and labeling bias in training data."""
    report = {}

    for col in sensitive_cols:
        group_counts = df[col].value_counts(normalize=True)

        # 1. Representation bias: is any group underrepresented?
        min_rep = group_counts.min()
        report[f"{col}_representation_gap"] = group_counts.max() - min_rep

        # 2. Label distribution by group
        label_by_group = df.groupby(col)[target_col].mean()
        report[f"{col}_label_rate"] = label_by_group.to_dict()

        # 3. Stereotypical correlations that might cause proxy discrimination
        for other_col in df.select_dtypes(include='object').columns:
            if other_col not in [col, target_col]:
                cramers_v = compute_cramers_v(df[col], df[other_col])
                if cramers_v > 0.3:
                    print(f"Strong association: {col} <-> {other_col} (V={cramers_v:.2f})")
                    print(f"   Risk: {other_col} may act as proxy for {col}")

    # Summary
    print("\n=== DATASET BIAS AUDIT ===")
    for key, val in report.items():
        print(f"{key}: {val}")

def compute_cramers_v(x, y):
    """Cramér's V — measure of association between two categorical variables."""
    from scipy.stats import chi2_contingency
    confusion_matrix = pd.crosstab(x, y)
    chi2 = chi2_contingency(confusion_matrix)[0]
    n = confusion_matrix.sum().sum()
    phi2 = chi2 / n
    r, k = confusion_matrix.shape
    return np.sqrt(phi2 / min(k-1, r-1))

Debiasing Techniques

Technical debiasing interventions fall into three stages, each with distinct advantages and limitations. Pre-processing methods modify the training data before any model is trained; they are model-agnostic but cannot prevent a sufficiently powerful model from learning proxy representations of protected attributes from correlated features. In-processing methods incorporate fairness objectives directly into the training procedure; they typically achieve better accuracy-fairness trade-offs but require more engineering. Post-processing methods adjust predictions after the model is trained, treating it as a black box; they are easy to apply but can only reshape decision boundaries rather than change what the model has learned.

Pre-processing Methods

The simplest pre-processing interventions address representation imbalances directly. Resampling either oversamples underrepresented demographic groups (using SMOTE for tabular data or augmentation for images) or undersamples overrepresented groups. Reweighting assigns higher loss weights to samples from underrepresented groups, achieving similar effects without discarding data. The IBM AIF360 Disparate Impact Remover applies a more principled transformation: it modifies the feature values of minority-group instances to reduce statistical correlation between features and the protected attribute while preserving relative ranking within each group.

The following implementation computes per-group-per-label sample weights to equalise the joint distribution, reducing disparate impact without discarding data:

from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
import numpy as np

def compute_fairness_weights(y_true, sensitive_attr):
    """Pre-processing: reweight training samples to equalize group representation."""
    groups = np.unique(sensitive_attr)
    labels = np.unique(y_true)

    weights = np.ones(len(y_true))
    n_total = len(y_true)

    for group in groups:
        for label in labels:
            mask = (sensitive_attr == group) & (y_true == label)
            n_group_label = mask.sum()
            n_group = (sensitive_attr == group).sum()
            n_label = (y_true == label).sum()

            if n_group_label > 0:
                # Expected count if independent (unbiased)
                expected = (n_group / n_total) * (n_label / n_total) * n_total
                # Weight = expected / actual (upweight underrepresented, downweight overrepresented)
                weights[mask] = expected / n_group_label

    return weights / weights.mean()  # normalize

# Unbiased training
sample_weights = compute_fairness_weights(y_train, gender_train)
model = LogisticRegression()
model.fit(X_train_scaled, y_train, sample_weight=sample_weights)

# Before reweighting: DI ratio = 0.67 (violation)
# After reweighting: DI ratio = 0.83 (compliant)
# Accuracy cost: 94.2% → 91.8% (small tradeoff for fairness)

In- & Post-processing Methods

Adversarial debiasing is the most prominent in-processing approach: a classifier is trained to predict the target label while simultaneously minimising an adversary's ability to predict the protected attribute from the classifier's internal representations. The adversary acts as a regulariser, penalising the classifier for learning protected-attribute-predictive features. Fairlearn's exponentiated gradient reduction reformulates fairness constraints as Lagrangian multipliers and optimises a sequence of reweighted classifiers to find the accuracy-fairness Pareto frontier — the set of models at which no fairness improvement is possible without a corresponding accuracy loss.

Post-processing methods are model-agnostic and operationally simpler. Calibrated equalised odds post-processing searches for the optimal decision thresholds per demographic group to satisfy an equalised odds constraint, allowing a different threshold for each group. This is effective but carries legal risk in some jurisdictions: applying different thresholds to different demographic groups may constitute disparate treatment under anti-discrimination law. The reject option classifier introduces an abstention zone around the decision boundary and routes borderline cases from the disadvantaged group to a human reviewer, reducing the model's footprint in the highest-uncertainty region.

Participatory & Values-Based Design

Technical debiasing is a necessary condition for equitable AI, but it is not a sufficient one. The most significant fairness failures in deployed AI systems have arisen not from inadequate algorithms but from inadequate problem framing — from decisions made before a single line of model code was written. Which protected attributes to measure, which fairness criterion to optimise, which harms are in scope, and whether the automated system should exist at all in the given context: none of these are questions that a metric or an algorithm can answer. They are questions about values, power, and whose interests the system should serve. Answering them well requires including the communities most likely to be affected by the system in the design process — not as users to be consulted after the fact but as stakeholders whose knowledge is necessary for adequate problem formulation.

Participatory design methods — co-design workshops, community advisory boards, ethnographic field studies, and iterative prototyping with community representatives — create structured processes for this inclusion. Value-sensitive design provides a methodology for systematically identifying the values at stake in a technology design, the stakeholders who hold those values, and the design choices that support or undermine them. These approaches have surfaced problems that aggregate fairness metrics missed: public sector child welfare screening tools in the United States satisfied standard group fairness criteria while still systematically over-surveilling low-income families — harms that only emerged through qualitative engagement with affected populations.

The broader point is that fairness is not a property of a model in isolation; it is a property of a sociotechnical system that includes the model, the people who use it, the institutions that deploy it, and the communities it affects. A model that is statistically fair by one metric can still be deployed in a context that produces discriminatory outcomes — through biased human interpretation of its outputs, through selection effects in who interacts with the system, or through the power dynamics that determine who bears the costs when the system errs.

Exercises

Regulatory Landscape & Legal Frameworks

AI fairness is not only an engineering and ethical obligation — it is increasingly a legal one. The regulatory landscape for algorithmic discrimination is complex, jurisdiction-specific, and rapidly evolving. Understanding the legal framework that governs AI decision-making in your deployment context is a prerequisite for compliance, not a post-hoc concern. The major regulatory frameworks fall into three categories: anti-discrimination law applied to algorithmic contexts, sector-specific AI regulations, and emerging horizontal AI governance frameworks.

Anti-Discrimination Law Applied to AI

In the United States, Title VII of the Civil Rights Act prohibits employment discrimination on the basis of race, color, religion, sex, and national origin. The Equal Employment Opportunity Commission (EEOC) has confirmed that this applies to AI-assisted hiring decisions: if an algorithm produces a disparate impact on a protected class — that is, if the selection rate for a protected group is less than 80% of the selection rate for the most-favoured group (the "four-fifths rule" or 80% rule) — the employer must demonstrate that the selection criterion is job-related and consistent with business necessity. This is precisely the disparate impact ratio that the compute_fairness_metrics function above computes. The Equal Credit Opportunity Act (ECOA) and Fair Housing Act similarly prohibit algorithmic discrimination in credit and housing decisions. The Federal Trade Commission Act's prohibition on unfair or deceptive practices has been applied to opaque algorithmic systems, most prominently in the FTC's 2021 guidance on AI and algorithmic decision-making.

In the European Union, the General Data Protection Regulation (GDPR) Article 22 grants individuals the right not to be subject to solely automated decisions that produce legal or similarly significant effects, with exceptions for necessity, contract, or explicit consent. When these exceptions apply, individuals have the right to obtain human review, to express their point of view, and to contest the decision. The GDPR also requires that profiling systems be accompanied by meaningful information about the logic involved. The EU AI Act (2024, effective 2026) goes further: it designates AI systems used in high-risk contexts (employment, credit, education, essential services, law enforcement) as "high-risk AI systems" subject to mandatory conformity assessments, bias testing, technical documentation, and registration before deployment. High-risk AI systems must use representative training data, be monitored for discrimination in production, provide explainability to affected individuals, and be overseen by a "responsible person."

Case Studies: When Fairness Auditing Made a Difference

The value of systematic fairness auditing is demonstrated most clearly by cases where it discovered problems that standard evaluation would have missed entirely.

In-Processing Debiasing with Fairlearn

Pre-processing removes bias before the model sees data; post-processing adjusts predictions after training. In-processing debiasing incorporates fairness constraints directly into the training objective, allowing the optimiser to explore the full accuracy-fairness Pareto frontier. Fairlearn's ExponentiatedGradient meta-algorithm wraps any scikit-learn compatible estimator and enforces demographic constraints through Lagrangian optimisation over a sequence of reweighted training runs.

from fairlearn.reductions import ExponentiatedGradient, EqualizedOdds, DemographicParity
from fairlearn.metrics import MetricFrame, demographic_parity_difference
from fairlearn.metrics import equalized_odds_difference
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, balanced_accuracy_score
from sklearn.model_selection import train_test_split
import pandas as pd
import numpy as np

# ── 1. Baseline model (unconstrained) ──────────────────────────────────────
X_train, X_test, y_train, y_test, A_train, A_test = train_test_split(
    X, y, sensitive_feature, test_size=0.2, random_state=42, stratify=y
)

baseline = LogisticRegression(max_iter=1000)
baseline.fit(X_train, y_train)
y_pred_base = baseline.predict(X_test)

print("=== Baseline Model ===")
print(f"Overall accuracy:  {accuracy_score(y_test, y_pred_base):.4f}")
print(f"DP difference:     {demographic_parity_difference(y_test, y_pred_base, sensitive_features=A_test):.4f}")
print(f"EO difference:     {equalized_odds_difference(y_test, y_pred_base, sensitive_features=A_test):.4f}")

# ── 2. MetricFrame: per-group breakdown ────────────────────────────────────
from sklearn.metrics import recall_score, precision_score
metrics_dict = {
    'accuracy': accuracy_score,
    'precision': lambda y, yp: precision_score(y, yp, zero_division=0),
    'recall': recall_score,
}
mf_base = MetricFrame(
    metrics=metrics_dict,
    y_true=y_test,
    y_pred=y_pred_base,
    sensitive_features=A_test
)
print("\nPer-group metrics (baseline):")
print(mf_base.by_group.to_string())
print(f"Max disparity (recall): {mf_base.difference()['recall']:.4f}")

# ── 3. ExponentiatedGradient with EqualizedOdds constraint ────────────────
constraint = EqualizedOdds(difference_bound=0.05)  # allow up to 5% disparity
mitigated_clf = ExponentiatedGradient(
    estimator=LogisticRegression(max_iter=1000),
    constraints=constraint,
    eps=0.01,      # constraint tolerance
    nu=1e-6,       # optimality tolerance
    max_iter=50,   # Lagrangian update iterations
)
mitigated_clf.fit(X_train, y_train, sensitive_features=A_train)
y_pred_mitigated = mitigated_clf.predict(X_test)

print("\n=== Mitigated Model (EqualizedOdds, bound=0.05) ===")
print(f"Overall accuracy:  {accuracy_score(y_test, y_pred_mitigated):.4f}")
print(f"DP difference:     {demographic_parity_difference(y_test, y_pred_mitigated, sensitive_features=A_test):.4f}")
print(f"EO difference:     {equalized_odds_difference(y_test, y_pred_mitigated, sensitive_features=A_test):.4f}")

mf_mitigated = MetricFrame(
    metrics=metrics_dict,
    y_true=y_test,
    y_pred=y_pred_mitigated,
    sensitive_features=A_test
)
print("\nPer-group metrics (mitigated):")
print(mf_mitigated.by_group.to_string())

# ── 4. Pareto sweep across DemographicParity bounds ───────────────────────
results = []
for eps in [0.0, 0.02, 0.05, 0.10, 0.20]:
    clf = ExponentiatedGradient(
        LogisticRegression(max_iter=500),
        constraints=DemographicParity(difference_bound=eps),
        max_iter=30
    )
    clf.fit(X_train, y_train, sensitive_features=A_train)
    yp = clf.predict(X_test)
    results.append({
        'dp_bound': eps,
        'accuracy': accuracy_score(y_test, yp),
        'dp_diff': demographic_parity_difference(y_test, yp, sensitive_features=A_test),
        'eo_diff': equalized_odds_difference(y_test, yp, sensitive_features=A_test),
    })

pareto_df = pd.DataFrame(results)
print("\nAccuracy–Fairness Pareto frontier:")
print(pareto_df.to_string(index=False, float_format='{:.4f}'.format))

Post-Processing: Calibrated Equalized Odds

Post-processing debiasing adjusts the decision threshold independently per demographic group to satisfy a fairness constraint. Hardt et al. (2016) introduced the equalized odds post-processing method: given a base classifier's predicted probabilities, find group-specific thresholds that minimise accuracy loss subject to equal TPR and FPR across groups. This approach requires access to group membership at prediction time and does not modify the model itself — making it deployable as a scoring wrapper around any existing production model.

Fairness Monitoring in Production

A fairness audit performed at training time is a snapshot — it reflects the distribution of the training and evaluation data at one point in time. Production fairness requires continuous monitoring because the relationship between model outputs and demographic groups can change as the user population shifts, as upstream data pipelines evolve, and as the policy context the model operates within changes. The MLOps monitoring infrastructure (covered in Part 20) needs to track demographic slice metrics alongside aggregate performance metrics with equivalent alerting severity.

Slice-Based Monitoring Architecture

Slice-based monitoring disaggregates model evaluation metrics by sensitive attribute values (and their intersections) and computes disparity metrics continuously against a production baseline established at deployment time. The alert hierarchy should mirror that used for overall performance: a warning-level alert for fairness metric creep (gradual worsening over weeks), an error-level alert for sudden fairness regression (single-day spike), and an incident-level alert for absolute violation of the deployed fairness standard.

Fairness Documentation Standards

Systematic documentation of fairness properties, training data composition, and intended use is not optional in the current regulatory environment. Two key standards have emerged as de facto requirements:

• Datasheets for Datasets (Gebru et al., 2018): A structured questionnaire covering motivation, composition, collection process, preprocessing transformations, uses, distribution, and maintenance of training datasets. Completing a datasheet surfaces the decisions and constraints that determine what biases are encoded in the data.
• Model Cards (Mitchell et al., 2019): A standardised report accompanying model releases that documents: model details (architecture, training procedure), intended use and out-of-scope uses, factors (demographic, environmental, instrumental variables that affect performance), metrics (performance quantified per subgroup), evaluation data, training data, quantitative analyses, and ethical considerations. The EU AI Act's technical documentation requirements for high-risk AI systems closely mirror the Model Card structure.

Both documents should be versioned and updated with every model version release. Model cards are increasingly required by deployment platforms: HuggingFace Model Hub requires a model card for all public models, and several US federal agencies now require model cards for AI systems used in high-stakes decisions. The EU AI Act Article 13 (transparency obligations) and Annex IV (technical documentation) effectively mandate Model Card-equivalent documentation for all high-risk AI systems placed on the EU market from August 2026.

Adversarial Debiasing: Neural In-Processing

Adversarial debiasing trains two networks simultaneously: a predictor that tries to maximise task performance, and an adversary that tries to predict the sensitive attribute from the predictor's representation or output. The predictor is trained to fool the adversary — i.e., to produce representations from which the sensitive attribute cannot be inferred — while maintaining task accuracy. When the adversary cannot predict group membership better than the base rate, the representation is approximately independent of the sensitive attribute, satisfying an information-theoretic form of demographic parity.

import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader, TensorDataset

# ── Architecture ──────────────────────────────────────────────────────────
class Predictor(nn.Module):
    """Predicts label Y from features X; produces intermediate representation."""
    def __init__(self, input_dim, hidden_dim=128, repr_dim=64):
        super().__init__()
        self.encoder = nn.Sequential(
            nn.Linear(input_dim, hidden_dim),
            nn.ReLU(),
            nn.Dropout(0.2),
            nn.Linear(hidden_dim, repr_dim),
            nn.ReLU(),
        )
        self.classifier = nn.Linear(repr_dim, 1)  # binary classification

    def forward(self, x):
        representation = self.encoder(x)
        logit = self.classifier(representation)
        return logit, representation

class Adversary(nn.Module):
    """Tries to predict sensitive attribute A from predictor representation."""
    def __init__(self, repr_dim=64, hidden_dim=32, num_groups=1):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(repr_dim, hidden_dim),
            nn.ReLU(),
            nn.Linear(hidden_dim, num_groups),  # binary or multi-class
        )

    def forward(self, representation):
        return self.net(representation)

# ── Training Loop ─────────────────────────────────────────────────────────
def train_adversarial_debiasing(
    X_train, y_train, A_train,
    n_epochs=50,
    alpha=1.0,      # adversary loss weight (tune: higher = more fairness, less accuracy)
    lr_pred=1e-3,
    lr_adv=1e-3,
):
    input_dim = X_train.shape[1]
    dataset = TensorDataset(
        torch.FloatTensor(X_train.values),
        torch.FloatTensor(y_train.values),
        torch.FloatTensor(A_train.values),
    )
    loader = DataLoader(dataset, batch_size=256, shuffle=True)

    predictor = Predictor(input_dim=input_dim)
    adversary  = Adversary(repr_dim=64)

    opt_pred = optim.Adam(predictor.parameters(), lr=lr_pred)
    opt_adv  = optim.Adam(adversary.parameters(), lr=lr_adv)
    bce = nn.BCEWithLogitsLoss()

    for epoch in range(n_epochs):
        predictor.train(); adversary.train()
        total_pred_loss = total_adv_loss = 0.0

        for X_batch, y_batch, A_batch in loader:
            # ── Step 1: Update adversary (maximise its prediction of A) ───
            opt_adv.zero_grad()
            with torch.no_grad():
                _, repr_batch = predictor(X_batch)
            adv_logit = adversary(repr_batch)
            adv_loss = bce(adv_logit.squeeze(), A_batch)
            adv_loss.backward()
            opt_adv.step()

            # ── Step 2: Update predictor (task loss - alpha * adversary loss)
            opt_pred.zero_grad()
            pred_logit, repr_batch = predictor(X_batch)
            task_loss = bce(pred_logit.squeeze(), y_batch)

            adv_logit = adversary(repr_batch)
            adv_confusion_loss = bce(adv_logit.squeeze(), A_batch)

            # Predictor minimises task loss but MAXIMISES adversary loss
            pred_loss = task_loss - alpha * adv_confusion_loss
            pred_loss.backward()
            opt_pred.step()

            total_pred_loss += task_loss.item()
            total_adv_loss  += adv_confusion_loss.item()

        if (epoch + 1) % 10 == 0:
            print(f"Epoch {epoch+1:3d} | task_loss={total_pred_loss/len(loader):.4f} | adv_loss={total_adv_loss/len(loader):.4f}")

    return predictor, adversary

predictor, adversary = train_adversarial_debiasing(
    X_train, y_train, A_train, n_epochs=50, alpha=2.0
)

# ── Evaluate fairness of debiased representations ─────────────────────────
predictor.eval()
with torch.no_grad():
    pred_logits, representations = predictor(torch.FloatTensor(X_test.values))

y_pred_adv = (torch.sigmoid(pred_logits).squeeze() > 0.5).numpy().astype(int)

from fairlearn.metrics import demographic_parity_difference, equalized_odds_difference
from sklearn.metrics import accuracy_score

print(f"Accuracy:     {accuracy_score(y_test, y_pred_adv):.4f}")
print(f"DP diff:      {demographic_parity_difference(y_test, y_pred_adv, sensitive_features=A_test):.4f}")
print(f"EO diff:      {equalized_odds_difference(y_test, y_pred_adv, sensitive_features=A_test):.4f}")

# Test whether the adversary can still predict group from representations
from sklearn.linear_model import LogisticRegression
repr_np = representations.numpy()
adv_probe = LogisticRegression().fit(repr_np[:int(0.8*len(repr_np))], A_test[:int(0.8*len(A_test))])
adv_acc = adv_probe.score(repr_np[int(0.8*len(repr_np)):], A_test[int(0.8*len(A_test)):])
print(f"Adversary probe accuracy on representations: {adv_acc:.4f}")
print(f"(Near 0.5 = sensitive attribute not recoverable from representation)")

Participatory Design and Stakeholder Engagement

Technical fairness metrics can identify distributional disparities, but they cannot identify which disparities are ethically significant, which harms are most severe, or which stakeholder groups bear the heaviest burden of system errors. Participatory design — involving affected communities in system design, evaluation, and governance — addresses the values gap in purely technical approaches to fairness.

Structured Engagement Frameworks

Meaningful participation requires more than consulting focus groups or running surveys. Three levels of engagement are distinguishable:

• Consultation: Sharing design decisions with affected groups and inviting feedback. Minimum viable participation — necessary but not sufficient for systems with significant harm potential.
• Co-design: Affected community representatives participate in defining the problem formulation, selecting fairness criteria, designing the evaluation methodology, and reviewing audit findings. This level is appropriate for high-stakes systems affecting vulnerable populations.
• Community ownership: The affected community controls key governance decisions: deployment scope, audit scheduling, override authority for individual decisions, and sunset criteria. Appropriate for systems that affect communities with historical experience of institutional harm from automated decision-making (e.g., predictive policing in communities with documented police discrimination).

The choice of fairness criterion should always be validated through stakeholder engagement. There is no technical basis for choosing equalised odds over demographic parity — the choice reflects a value judgment about what kind of fairness matters most in the specific context, and the people most affected by that choice have the strongest claim to participate in making it.

Conclusion & Next Steps

Bias in AI systems is multi-causal and multi-stage. It enters through historical inequities encoded in data, through representation gaps in training sets, through label quality failures, through model objectives that optimise aggregate performance at the expense of minority groups, and through deployment contexts that amplify rather than correct for structural disparities. No single intervention is sufficient: pre-processing, in-processing, and post-processing debiasing techniques each address different failure modes and none of them substitutes for careful dataset auditing, formal documentation through datasheets and model cards, and ongoing fairness monitoring in production.

The impossibility theorems are a crucial anchor for practitioners: they establish that choosing a fairness criterion is unavoidable, that no criterion is universally correct, and that the choice carries real distributional consequences for the people subject to the system. Making that choice explicitly, documenting the reasoning, and revisiting it as deployment conditions change is the operational reality of responsible AI development. A system that is acceptably fair today may become unfair as the population it serves shifts, as the policies it informs change, or as the training data grows stale — which is why fairness monitoring must be a continuous practice, not a one-time audit.

The MLOps infrastructure we cover in Part 20 — specifically model monitoring, drift detection, and automated alerting — provides the technical foundation for that continuity. The organisational and regulatory dimensions are addressed in Part 23 on Responsible AI Governance.