Bias, fairness, and discrimination metrics: Question bank

Before we start, one shared vocabulary reminder so no symbol is used unearned:

The confusion matrix (every symbol earned first)

Every group-fairness metric below is a fraction built from four counts. Picture a $2 \times 2$ table: the columns say what actually happened ( $Y$ ), the rows say what the model guessed ( $\hat{Y}$ ). The figure below is a self-contained walkthrough — read it top to bottom before the definitions.

The layout, spelled out so it works even if the image does not render:

Two columns = the truth $Y$ . Left column (teal) = "actually positive" $Y = 1$ ; right column (plum) = "actually negative" $Y = 0$ .
Two rows = the prediction $\hat{Y}$ . Top row (orange edge) = "predicted positive" $\hat{Y} = 1$ ; bottom row = "predicted negative" $\hat{Y} = 0$ .
The four cells are then just the four ways truth and guess can meet: top-left $T P$ (orange), top-right $F P$ (plum), bottom-left $F N$ (teal), bottom-right $T N$ (tan).
The coloured arrows show the direction each metric reads the table: TPR and FPR run down a column (fixing the truth), while precision runs across the top row (fixing the prediction).

Figure — Bias, fairness, and discrimination metrics

Read the conditioning bars carefully: TPR/FPR condition on the truth $Y$ (columns), while precision conditions on the prediction $\hat{Y}$ (rows). This flip is where half the traps below live.

Formula The Chouldechova identity (the engine of the conflict) — every step shown

Step 1 (WHAT). Split the flag rate by the law of total probability, since every predicted-positive is either a true or a false positive: $P (\hat{Y} = 1 ∣ A = a) = true positives TPR \cdot b + false positives FPR \cdot (1 - b)$ Step 2 (WHAT). Write precision as true-positives over all predicted-positives, using that same split in the denominator: $PPV = \frac{TPR \cdot b}{TPR \cdot b + FPR \cdot ( 1 - b )}$ Step 3 (WHY — solve for FPR). We want FPR alone to see how errors relate to base rate and precision. Cross-multiply by the denominator: $PPV \cdot (TPR \cdot b + FPR \cdot (1 - b)) = TPR \cdot b$ Step 4. Expand and move the FPR term to one side, everything else to the other: $PPV \cdot FPR \cdot (1 - b) = TPR \cdot b - PPV \cdot TPR \cdot b = TPR \cdot b (1 - PPV)$ Step 5. Divide both sides by $PPV \cdot (1 - b)$ to isolate FPR: $FPR = \frac{b}{1 - b} \cdot \frac{1 - PPV}{PPV} \cdot TPR$ Why this matters: impose equalized odds, so $TPR$ and $FPR$ match across groups. The identity then forces the base-rate factors $\frac{b}{1 - b}$ to carry the $PPV$ terms. If $b_{0} \neq = b_{1}$ , then $PPV_{0} \neq = PPV_{1}$ — equal errors force unequal precision. That is the impossibility.

The second figure makes the bound concrete. On the horizontal axis is $d_{X}$ (how far apart two applicants are); on the vertical axis is $d_{Y}$ (how far apart their predictions are). The straight line $d_{Y} = L \cdot d_{X}$ is the ceiling — every fair pair must sit on or below it. The green dot (tiny $d_{X}$ , tiny $d_{Y}$ ) obeys the rule; the red dot (tiny $d_{X}$ but a huge $d_{Y}$ jump) pokes above the ceiling and flags a fairness violation.

True or false — justify

Demographic parity requires the model to ignore the protected attribute

A

as an input feature.

False — parity is the output condition

P (\hat{Y} = 1 ∣ A = 0) = P (\hat{Y} = 1 ∣ A = 1)

; you can even satisfy it by using

A

during training to actively balance rates. "Fairness through unawareness" (dropping

A

) is a different, weaker idea.

If a model never receives race as a feature, it cannot be racially biased.

False — the model learns proxy variables (ZIP code, purchase history) that correlate with race, so discrimination sneaks back in. This is exactly why disparate impact tests outcomes, not inputs.

Equalized odds and predictive parity can always be satisfied together with enough model tuning.

False — the Chouldechova identity

FPR = \frac{b}{1 - b} \cdot \frac{1 - PPV}{PPV} \cdot TPR

forces a tradeoff whenever base rates differ and the classifier is imperfect; no tuning escapes an algebraic identity.

Equal opportunity is just another name for equalized odds.

False — equal opportunity equalizes only

TPR

(the

Y = 1

column); equalized odds also equalizes

FPR

(the

Y = 0

column). Equal opportunity is the strictly weaker "half" of equalized odds.

A perfectly accurate classifier is automatically fair under every metric.

False — a perfect classifier sets

\hat{Y} = Y

, so its flag rate equals each group's base rate

b

; if

b_{0} \neq = b_{1}

it violates demographic parity by construction.

Demographic parity is a statement about

\hat{Y}

and

A

only, never about

Y

True —

P (\hat{Y} = 1 ∣ A)

conditions the prediction on the group and looks at nothing else, so it can be measured on unlabelled data. That is also its weakness: it ignores whether the prediction was correct.

A disparate impact ratio of exactly

1.0

proves the model is fair.

False — it proves equal selection rates (demographic parity), but says nothing about equalized odds (

TPR / FPR

) or precision. Different metrics catch different unfairness.

Individual fairness (the Lipschitz bound

d_{Y} (f (X_{i}), f (X_{j})) \leq L d_{X} (X_{i}, X_{j})

) implies group fairness.

False — treating every pair of similar individuals consistently can still produce group-level rate gaps if the groups genuinely differ in the measured features. The two operate at different scales.

Calibration (predictive parity) being satisfied means the two groups have equal false positive rates.

False — calibration equalizes

PPV = P (Y = 1 ∣ \hat{Y} = 1)

; FPR is the opposite conditioning

P (\hat{Y} = 1 ∣ Y = 0)

. Equalizing one does not equalize the other when base rates differ.

Spot the error

"Group A is favourably treated 60%, Group B 40%, so disparate impact

= \frac{0.60}{0.40} = 1.5

, which is fine."

Wrong orientation — the lower-rate (disadvantaged) group goes on top:

\frac{0.40}{0.60} = 0.67 < 0.8

, which fails. B has the smaller favourable rate, so B is disadvantaged; flipping the ratio hides that.

P (\hat{Y} = 1 ∣ A = 0)

T P

for group 0, divided by group 0 size."

Error — the numerator must be all predicted positives,

T P + F P

(the whole

\hat{Y} = 1

row total), not just

T P

. Demographic parity never touches the true label

Y

"TPR is

\frac{T P}{T P + F P}

because both are things we predicted positive."

Error — that ratio is precision (

PPV

). TPR is

\frac{T P}{T P + F N}

: it conditions on

Y = 1

(the actual-positive column), asking "of those who truly were positive, how many did we catch?"

"Since equalized odds equalizes

TPR

and

FPR

, and precision is built from

T P

and

F P

, precision must also be equal."

Error — precision also depends on the base rate

b

via

FPR = \frac{b}{1 - b} \cdot \frac{1 - PPV}{PPV} \cdot TPR

. Equal

TPR / FPR

with unequal

b

forces unequal

PPV

"Equal opportunity guarantees the model makes no false alarms for the disadvantaged group."

Error — equal opportunity only equalizes

TPR

; it says nothing about

FPR

, so false alarms (false positives) can still pile up unequally. You need the FPR half of equalized odds for that.

"The 80% rule is a hard mathematical proof of discrimination."

Error — it is an empirical screening threshold (U.S. EEOC guideline), meant to flag cases for closer review, not a sharp legal or statistical boundary. Ratios near

0.8

need context.

"Base rate

b = P (Y = 1 ∣ A = a)

is something the model outputs."

Error —

b

is a property of the world / data, measured from true labels

Y

before any model exists. Confusing it with the model's flag rate

P (\hat{Y} = 1 ∣ A = a)

collapses the whole impossibility argument.

"In the loan audit,

\hat{Y} = 1

means the loan was approved."

Error — in that scenario

\hat{Y} = 1

means "predicted to default", which leads to denial. Sign confusion here flips every fairness conclusion, so always pin down which outcome is the favourable one.

Why questions

Why do we need multiple fairness definitions instead of one?

Because fairness is context-dependent: a statistician's "fair" (equal error rates, equalized odds) and a sociologist's "fair" (equal outcome rates, demographic parity) are genuinely different goals that cannot generally coexist. See Fairness-accuracy tradeoffs, where matching one metric provably costs accuracy or another metric.

Why does equal opportunity exist as a separate metric between the two extremes?

Because sometimes we only care that deserving people are not missed (equal

TPR

) but tolerate different false-alarm rates; relaxing the FPR constraint of equalized odds makes it easier to satisfy alongside good accuracy. It targets the harm of denied opportunity specifically.

Why does the impossibility result require imperfect classifiers to be interesting?

A perfect classifier trivially satisfies error-based metrics (

\hat{Y} = Y

) but can never equalize flag rates when base rates

b

differ; the real-world tension in the Chouldechova identity appears precisely because practical classifiers have nonzero

FPR

Why does the base-rate factor

\frac{b}{1 - b}

drive the whole conflict?

It is the algebraic bridge in the Chouldechova identity linking

FPR

TPR

, and

PPV

. When it differs between groups (

1.0

for

b = 0.5

\approx 0.43

for

b = 0.3

), holding

TPR

and

FPR

equal forces

PPV

to differ.

Why is "similar individuals get similar predictions" written as the Lipschitz bound

d_{Y} (f (X_{i}), f (X_{j})) \leq L d_{X} (X_{i}, X_{j})

Because it borrows analysis's smoothness idea: the output distance is capped by the input distance scaled by

L

, so tiny, irrelevant feature differences (a ZIP-code tweak with small

d_{X}

) cannot cause large prediction swings.

Why does causal reasoning (Counterfactual fairness / Causal fairness) go beyond these statistical metrics?

Statistical metrics only see correlations in observed

\hat{Y}, Y, A

; a causal view asks "would the decision change if this person's

A

had been different, all else equal?", catching bias flowing through proxy pathways the correlational metrics miss. This underpins Algorithmic accountability audits.

Why can two groups' precision differ even when the model is "equally good" at ranking?

Because

PPV

mixes in the prevalence

b

: the same score threshold applied to a lower-

b

group yields relatively more false positives, dragging

PPV

down — a base-rate effect, not a ranking flaw.

Edge cases

What happens to demographic parity when the two groups have identical base rates

b_{0} = b_{1}

Then equalized odds and predictive parity can be satisfied together and demographic parity aligns naturally — the impossibility conflict vanishes because the factor

\frac{b}{1 - b}

is equal on both sides of the Chouldechova identity.

What does the disparate impact ratio equal if the privileged group is favourably treated

0%

of the time?

The ratio is

\frac{something}{0}

, undefined — a degenerate case. A zero-rate denominator signals the test itself has broken down; inspect raw counts rather than the ratio. Relevant to real COMPAS risk assessment audits with sparse subgroups.

If a group has zero actual positives (

Y = 1

never occurs), can we compute its TPR or equal opportunity?

No —

TPR = \frac{T P}{T P + F N}

has denominator

T P + F N = 0

, so it is undefined; both equal opportunity and equalized odds simply cannot be evaluated on that column for that group.

What if a person belongs to two disadvantaged groups at once (e.g. gender and race)?

Single-axis metrics can look fair on each axis separately yet hide severe unfairness at the intersection — the core warning of Intersectionality in ML. Auditing marginals is not enough; audit the joint subgroups.

In the limit

L \to \infty

in the Lipschitz bound, what does the constraint say?

Nothing — an infinite

L

allows arbitrarily large prediction jumps

d_{Y}

for tiny feature gaps

d_{X}

, so the fairness guarantee is vacuous. Small

L

is what makes the condition bite.

What does disparate impact of exactly

0.8

mean under the rule?

It sits on the screening boundary: not automatically a violation, but the edge of the "suggests discrimination" zone, so it warrants human review rather than a pass or fail. The rule is a soft flag, not a cliff.

If both groups are flagged at the same rate but errors concentrate entirely in one group, which metrics pass and fail?

Demographic parity passes (equal flag rates), but equalized odds fails because the true-positive and false-positive rates differ between groups, showing why equal outcomes do not guarantee equal treatment of errors. This is the classic COMPAS risk assessment-style scenario.

Recall One-line self-test

Which single quantity, differing between groups, mathematically forces the equalized-odds vs predictive-parity conflict? ::: The base rate $b = P (Y = 1 ∣ A = a)$ , entering through the factor $\frac{b}{1 - b}$ in the Chouldechova identity.