Foundations — Bias, fairness, and discrimination metrics

This page assumes you have seen none of the notation on the parent note. We build every piece from a picture of people and boxes. By the end you will be able to read every formula there without pausing.

Start with people, not symbols

Before any letter, picture a crowd of applicants. Each person is one dot. For each dot, two facts eventually become known:

1. What the model said about them ("we predict you will default" / "we predict you won't").
2. What actually happened later ("they did default" / "they didn't").

That is the entire universe of these metrics. Look at the board below: every fairness idea is a way of slicing this crowd.

Symbol 1 — $Y$ : the truth

Picture: in the crowd, colour every dot by truth — one colour for $Y=1$, another for $Y=0$. This colouring exists whether or not any model was ever built. It is reality.

Why the topic needs it: you cannot say a prediction is "right" or "wrong" without a truth to compare against. $Y$ is that truth.

Symbol 2 — $\hat{Y}$ : the guess

Picture: draw a box around the dots the model flagged. Inside the box: $\hat{Y}=1$. Outside: $\hat{Y}=0$. The box and the truth-colouring are two independent slicings of the same crowd — and that is the whole game.

Symbol 3 — $A$ : the group

Picture: split the entire crowd into two rooms — Room $A=0$ and Room $A=1$. We will run the same counting recipe inside each room and then compare the two answers. Fairness = "do the rooms get similar numbers?"

Now every dot carries three tags: its truth $Y$, its prediction $\hat{Y}$, and its room $A$.

The four boxes: TP, FP, FN, TN

When you overlay "truth colour" with "prediction box," each dot lands in exactly one of four boxes. This 2×2 grid is called the confusion matrix, and it is the single most important picture on this page.

Reading the words: the second word (Positive/Negative) is what the model said. The first word (True/False) is whether the model was right. So "False Positive" = "the positive claim was false."

Every person lands in exactly one box, so the four counts add up to the whole group: $N=TP+FP+FN+TN$ where $N$ is the total number of people in the group. Keep $N$ in mind — it is the "everyone" that fractions will divide by.

Why the topic needs it: every rate on the parent page — TPR, FPR, precision, flag-rate — is just one count divided by a total made of two counts. Learn the four boxes and every formula becomes arithmetic.

Symbol 4 — $P(\cdot )$ and $P(\cdot |\cdot )$ : "what fraction"

This is the one symbol beginners fear. It is simpler than it looks.

Example from the crowd: if $300$ of $1000$ people are flagged, then $P(\hat{Y}=1)=\frac{\text{count}(\hat{Y}=1)}{N}=\frac{300}{1000}=\mathrm{0.30.}$

Picture: the bar is a spotlight. $P(\text{box}\mid \text{condition})=\frac{\text{count}(\text{box and condition})}{\text{count}(\text{condition})}$ = walk into the room named by the condition, ignore everyone else, and count the box as a fraction of that room.

Let us now name every rate the parent uses as a $P$-with-a-bar, mapped straight to the four counts.

The named rates — each is one count over a two-count total

Symbol 5 — the base rate $b$

Picture: stand in one room, count only the truth-colour, ignore every prediction box. That fraction is $b$. It is a fact about the world, not about the model.

Why the topic needs it: the parent's impossibility proof hinges on $b_A\ne b_B$. When two groups differ in reality, the ratio-factor $\frac{b}{1-b}$ differs between them, and this is the wedge that forces the three fairness metrics apart. You cannot understand the Chouldechova identity without $b$.

Symbol 6 — $\forall$ and how it enters Equalized Odds

Watch it actually do its job. The parent's Equalized Odds condition is written compactly as $P(\hat{Y}=1\mid Y=y,A=0)=P(\hat{Y}=1\mid Y=y,A=1)\forall y\in \{0,1\}.$ The $\forall$ tells you to unfold this one line into two separate equations — substitute $y=1$, then $y=0$: $\underset{\text{equal TPR across groups }(y=1)}{\underbrace{P(\hat{Y}=1\mid Y=1,A=0)=P(\hat{Y}=1\mid Y=1,A=1)}}$ $\underset{\text{equal FPR across groups }(y=0)}{\underbrace{P(\hat{Y}=1\mid Y=0,A=0)=P(\hat{Y}=1\mid Y=0,A=1)}}$ So $\forall$ is exactly what packs "equal TPR and equal FPR" into a single tidy formula. Without it, you would have to write both lines by hand every time.

Symbol 7 — the ratio for Disparate Impact

A tiny worked check (positive outcome = approval): approve $60\%$ of A and $40\%$ of B, with B protected: $\text{DI}=\frac{0.40}{0.60}=0.667<0.8\Rightarrow \text{fails the screen.}$

Symbol 8 — distance $d(\cdot ,\cdot )$, the score $\hat{Y}$, and the constant $L$

Here $\hat{Y}$ switches domain as promised: it is now a real-valued score in $[0,1]$, not a $0/1$ flag. This is the raw confidence the model emits before thresholding.

Picture: imagine the model as a hill. $L$ is the maximum steepness allowed. If two people stand a hair apart on the input floor but the model launches one to a cliff-top score and drops the other to a valley, the hill was too steep — individual fairness is violated.

Worked check from the parent: feature gap $d(X_1,X_2)=0.01$, scores $0.72$ and $0.39$, so output gap $=0.33$. With $L=5$ the allowance is $5\times 0.01=0.05$. Since $0.33>0.05$, the constraint is broken.

How it all fits together

The map below is meant to be read as a story, not memorised. Walk it with me:

1. Three raw tags sit at the top — $Y$ (truth), $\hat{Y}$ (prediction), $A$ (group).
2. $Y$ and $\hat{Y}$ collide to make the four boxes (the confusion matrix); $A$ splits everyone into group rooms.
3. The conditional fraction $P(\cdot \mid \cdot )$ turns box-counts into rates inside each room.
4. Those rates split into two families: column-rates TPR/FPR and the row-rate PPV. Together with the base rate $b$ they feed the group-fairness metrics and the impossibility result.
5. On a separate track, $\hat{Y}$-as-a-score plus distance $d$ and constant $L$ feed individual fairness.
6. Everything empties into the parent topic.

Where these foundations lead

• The column-vs-row tension (TPR/FPR vs PPV) is exactly the machinery behind Fairness-accuracy tradeoffs and the impossibility result.
• The group rooms and flag-rate ratio underpin Disparate impact theory.
• The "similar inputs, similar outputs" idea generalises toward Counterfactual fairness and Causal fairness.
• Auditing the boxes in the real world is the practice of Algorithmic accountability, famously on COMPAS risk assessment.
• Layering multiple $A$ attributes at once leads to Intersectionality in ML.

Equipment checklist

Test yourself — cover the right side and answer aloud.