Worked examples — Confusion matrix interpretation

This is the "throw everything at it" companion to the parent note. There we derived the metrics. Here we stress-test them: every degenerate matrix, every extreme threshold, every sign of imbalance, and the word problems that trip people up. The rule of the page: guess the answer before you read the solution. If your guess is wrong, that cell of the scenario matrix is where your intuition needs repair.

Before we start, one shared picture we will point back to all page long.

The 2×2 grid: rows are reality (top row = actually positive, bottom = actually negative), columns are the model's guess (left = predicted positive, right = predicted negative). The green diagonal (TP, TN) is "the model was right." The coral off-diagonal (FN, FP) is "the model was wrong." Keep this grid in your head — every example below just fills these four boxes with different numbers, and each example tells you which box it is pushing to an extreme so you can locate it on this same picture.

The four metrics in one place (defined before we use them)

Everything below is built from four ratios. Read each as "correct count ÷ the group it belongs to." We anchor each denominator to a row or column of the grid above.

For a fuller treatment of these four ratios see Precision and Recall.

The scenario matrix

Here is the full menu of cases a confusion-matrix problem can be. Each worked example below is tagged with the cell(s) it covers, so you can check nothing is missed.

#	Case class	What is extreme	Which example
A	Perfect model	FP = FN = 0	Ex 1
B	Degenerate: predict-all-negative	TP = FP = 0	Ex 2
C	Degenerate: predict-all-positive	TN = FN = 0	Ex 3
D	Severe class imbalance	one row hugely bigger	Ex 2, Ex 4
E	Precision–recall tension	high P, low R (and reverse)	Ex 4, Ex 5
F	Threshold sweep (limiting values)	threshold → 0 and → 1	Ex 5 (+figure)
G	Zero-denominator traps	TP+FP = 0 or TP+FN = 0	Ex 6
H	Real-world word problem	translate story → matrix	Ex 7
I	Multi-class per-class extraction	one-vs-rest slicing	Ex 8
J	Exam twist: back-solve from metrics	given metrics, find cells	Ex 9

Prerequisites you may want open: Precision and Recall, Binary Classification Thresholds, Class Imbalance Handling, Multi-class Classification Metrics.

---

Example 1 — The perfect model (Case A)

Forecast: guess accuracy, precision, recall, specificity before reading. (Everyone gets this one — it is the anchor.)

1. Fill the grid. On the shared grid figure, every sample lands on the green diagonal: all positives in the top-left box (TP), all negatives in the bottom-right box (TN). $\text{TP}=120,\text{FN}=0,\text{FP}=0,\text{TN}=80$ Why this step? "Zero mistakes" means both coral off-diagonal boxes (FN, FP) are empty — the matrix is purely diagonal, the definition of a perfect classifier.

2. Compute metrics. $\text{Accuracy}=\frac{120+80}{200}=1.0$ $\text{Precision}=\frac{120}{120+0}=1.0,\text{Recall}=\frac{120}{120+0}=1.0,\text{Specificity}=\frac{80}{80+0}=1.0$ Why this step? Every metric is a ratio whose numerator equals its denominator when there are no errors, so all equal exactly $1$.

Verify: F1 harmonic mean of two $1$s is $1$. Diagonal-only matrix ⇒ every metric maxes out. Sanity: no metric can exceed $1$, and we hit the ceiling — consistent.

---

Example 2 — Predict "all negative" on imbalanced data (Cases B, D)

Forecast: accuracy will be high. Now guess recall — is it high or zero?

1. Fill the grid. On the shared grid, everything lands in the right column (predicted negative). Both left-column boxes empty out. $\text{TP}=0,\text{FN}=20,\text{FP}=0,\text{TN}=980$ Why this step? "Predicted positive" is the left column of our grid — if the model never says positive, that whole column is zero, which zeroes TP and FP.

2. Accuracy. $\text{Accuracy}=\frac{0+980}{1000}=0.98$ Why this step? The 980 legit emails are all correctly called negative; accuracy rewards this bulk and ignores the tiny positive class.

3. Recall. $\text{Recall}=\frac{0}{0+20}=0$ Why this step? Of the 20 real spam, the model caught none. Recall — the top-row view over actual positives — collapses to zero.

4. Precision. The denominator $\text{TP}+\text{FP}=0$ — undefined. See Example 6 for how to handle this; conventionally reported as $0$ or NaN.

Verify: accuracy $0.98$ vs recall $0$ — the exact "class-blind accuracy" trap from the parent's Mistake 1. A useless model scoring 98% proves accuracy alone is meaningless under imbalance. See Class Imbalance Handling.

---

Example 3 — Predict "all positive" (Case C)

Forecast: recall will be perfect. Guess what specificity does.

1. Fill the grid. On the shared grid, everything lands in the left column (predicted positive), so the right column empties out. $\text{TP}=20,\text{FN}=0,\text{FP}=980,\text{TN}=0$ Why this step? All 20 real spam are (accidentally) caught → TP $=20$, FN $=0$. All 980 legit are wrongly flagged → FP $=980$, TN $=0$.

2. Recall. $\text{Recall}=\frac{20}{20+0}=1.0$ Why this step? Catching everything guarantees you catch all positives — recall is trivially perfect. This is the "crying wolf" limit.

3. Precision. $\text{Precision}=\frac{20}{20+980}=\frac{20}{1000}=0.02$ Why this step? Of 1000 "spam" alarms, only 20 were real. Precision — the left-column view over predicted positives — is dismal.

4. Specificity. $\text{Specificity}=\frac{0}{0+980}=0$ Why this step? Of 980 real negatives, zero were correctly cleared. Perfect recall bought total loss of specificity.

Verify: compare with Ex 2 — Ex 2 has recall $0$/specificity $1$; Ex 3 has recall $1$/specificity $0$. The two degenerate extremes sit on opposite ends of the same seesaw. That seesaw is the ROC Curve and AUC itself.

---

Example 4 — A realistic imbalanced model (Cases D, E)

Forecast: accuracy near 98%. But guess precision — will it be high or low? (Watch the FP.)

1. Sanity-fill the grid. Top row (actual positives) $=70+30=100$ ✓, bottom row (actual negatives) $=200+9700=9900$ ✓, total $=10000$ ✓. Why this step? Always check the row sums match the stated class counts before computing anything.

2. Accuracy. $\text{Accuracy}=\frac{70+9700}{10000}=0.977$ Why this step? Sum the green diagonal (TP $+$ TN, the correct calls) over all 10000 predictions — accuracy is the plain "fraction right," and here the enormous TN dominates it into looking excellent.

3. Precision. $\text{Precision}=\frac{70}{70+200}=\frac{70}{270}\approx 0.259$ Why this step? 270 fraud alarms (the left column), only 70 real. Even a "good-looking" 97.7% model flags roughly 3 false alarms for every real fraud — precision exposes it.

4. Recall. $\text{Recall}=\frac{70}{70+30}=0.70$ Why this step? Of the 100 real frauds (the top row, TP $+$ FN), we caught 70 — recall reads the row over actual positives and reveals that 30 frauds slipped through.

5. Specificity. $\text{Specificity}=\frac{9700}{9700+200}\approx 0.980$ Why this step? High specificity because the huge negative bottom row swamps the 200 false alarms — a warning that specificity looks flattering under imbalance, unlike precision.

6. F1 (using the definition we set up above, with $P=$ precision, $R=$ recall). $F_1=\frac{2\cdot 70}{2\cdot 70+200+30}=\frac{140}{370}\approx 0.378$ Why this step? Using the closed form $F_1=\frac{2\text{TP}}{2\text{TP}+\text{FP}+\text{FN}}$ avoids rounding $P$ and $R$ first. $F_1$ sits low, between the low precision and moderate recall — the honest summary.

Verify: harmonic-mean check: $\frac{2PR}{P+R}=\frac{2(0.259)(0.70)}{0.259+0.70}\approx 0.378$ ✓, matches the closed form. Accuracy $0.977$ hid a model that is right only about 26% of the time when it shouts "fraud."

---

Example 5 — Threshold sweep: the limiting values (Cases E, F)

Forecast: at $t=1$ (predict positive almost never) and $t=0$ (predict positive always), what are $P$ and $R$?

The scores (positive samples marked ●, negatives ○), sorted high→low: $0.95_{●}0.90_{●}0.80_{○}0.70_{●}0.60_{○}0.55_{●}0.40_{○}0.30_{●}0.20_{○}0.10_{○}$

How to read this figure. The horizontal axis is the model's output score (from $0$ on the left to $1$ on the right). Each dot is one sample placed at its score; lavender dots are actual positives (marked $+$ above them), butter dots are actual negatives (marked $-$). A threshold $t$ is a vertical line: everything to the right of it (score $\ge t$) is predicted positive, everything to the left predicted negative. The two dashed lines show the two thresholds we test below. The top arrow reminds you that sliding $t$ leftward admits more samples as positive — this is the single knob that trades recall for precision.

1. Threshold $t=1.0$ (predict positive if score $\ge 1.0$). No dot is to the right of the far-right edge — nobody clears it. $\text{TP}=0,\text{FP}=0,\text{FN}=5\Rightarrow \text{Recall}=0,\text{Precision}=\frac{0}{0}(\text{undefined})$ Why this step? This is the right edge of the sweep — the same degenerate "predict-all-negative" state as Ex 2. Recall starts at $0$.

2. Threshold $t=0.72$ (the coral dashed line; score $\ge 0.72$ → positive). The three dots to its right are two lavender (●) and one butter (○). $\text{TP}=2,\text{FP}=1,\text{FN}=3\Rightarrow P=\frac{2}{3}\approx 0.667,R=\frac{2}{5}=0.40$ Why this step? Sliding the line left lets more samples through — recall rises, but the first negative sneaks in, so precision drops below $1$. This is the P–R tension made mechanical.

3. Threshold $t=0.50$ (the mint dashed line). Six dots now sit to its right: four ● and two ○. $\text{TP}=4,\text{FP}=2,\text{FN}=1\Rightarrow P=\frac{4}{6}\approx 0.667,R=\frac{4}{5}=0.80$ Why this step? Recall keeps climbing as we admit more positives; precision plateaus because we admitted one more of each class.

4. Threshold $t=0.0$ (predict positive always). Every dot is to the right. $\text{TP}=5,\text{FP}=5,\text{FN}=0\Rightarrow P=\frac{5}{10}=0.5,R=1.0$ Why this step? The left edge — the same "predict-all-positive" state as Ex 3. Recall maxes at $1$; precision falls to the base rate $\frac{5}{10}=0.5$.

Verify: recall is monotonically non-decreasing as $t$ drops ($0\to 0.40\to 0.80\to 1.0$) — correct, lowering a threshold can never lose a caught positive. Precision at $t=0$ equals the positive base rate $0.5$ — the classic limiting value. See Binary Classification Thresholds.

---

Example 6 — The zero-denominator traps (Case G)

Forecast: which metric blows up in each case, and what is the sensible convention?

1. Case (a): no actual positives. All 50 are negative and all are called negative: $\text{TP}=0,\text{FN}=0,\text{FP}=0,\text{TN}=50$. $\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}}=\frac{0}{0}(\text{undefined})$ Why this step? Recall asks "of the real positives, how many caught?" — but there are no real positives (top row is empty), so the question is meaningless. Convention: report recall as $1$ or NaN (nothing was there to miss). Specificity is fine: $\frac{50}{50}=1$.

2. Case (b): never predict positive, on a batch that has positives. Take a concrete batch of 50: suppose 8 are truly positive and 42 truly negative — these two numbers come from the batch's ground truth, not from us; we just need some positives present to expose the trap. Since the model never says positive, the entire left column is zero: $\text{TP}=0,\text{FP}=0$, and every real positive is missed so $\text{FN}=8$, every real negative is (passively) correct so $\text{TN}=42$. $\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}}=\frac{0}{0}(\text{undefined})$ Why this step? Precision asks "of your positive calls, how many right?" — but you made no positive calls (left column empty), so the denominator vanishes. The specific 8-and-42 split is irrelevant to the trap; any batch with at least one positive triggers it. Convention: precision $=0$ (you contributed no true positives) or NaN.

3. F1 with a zero. If either $P$ or $R$ is $0$, then $F_1=\frac{2PR}{P+R}=\frac{0}{P+R}=0(\text{if the denominator }P+R>0)$ Why this step? The harmonic mean rightly punishes: being perfect on one axis while zero on the other yields a zero $F_1$ — never $0.5$.

Verify: the "predict-all-negative" $F_1$ from Ex 2 is $\frac{2\cdot 0}{2\cdot 0+0+20}=0$, consistent with the rule here. Zero denominators are always a degenerate matrix, never a bug — name the missing group and move on.

---

Example 7 — Word problem: translate the story (Case H)

Forecast: the two questions map to two named metrics — which ones?

1. Identify positive class. Positive = "bag contains prohibited item" (the thing we want to detect). Why this step? Always define positive as the event of interest; every metric's meaning flips if you mislabel it.

2. Extract the four counts from the sentence.

   • "correctly stopped 45 of the 50" → TP $=45$, FN $=50-45=5$.
   • "also stopped 300 harmless" → FP $=300$.
   • remaining bags: total $5000$, threats $50$, so harmless $=4950$; correctly passed harmless $=4950-300=4650$ → TN $=4650$. Why this step? "Stopped" = predicted positive (left column); "harmless" = actually negative (bottom row). Cross the story's verbs with reality/prediction to fill the grid.

3. "How trustworthy is an alarm?" → precision. $\text{Precision}=\frac{45}{45+300}=\frac{45}{345}\approx 0.130$ Why this step? An "alarm" is a positive prediction (the left column); "how often is it right" is precision by definition.

4. "What fraction of threats slip through?" → miss rate $=1-\text{recall}$. $\text{Recall}=\frac{45}{45+5}=0.90\Rightarrow \text{slip-through}=1-0.90=0.10$ Why this step? "Slip through" = real positive predicted negative = FN out of all real positives = the complement of recall.

Verify: counts sum to $45+5+300+4650=5000$ ✓. Only 13% of alarms are real threats but only 10% of threats escape — a classic high-recall / low-precision security posture (see Cost-Sensitive Learning for why that trade is deliberate).

---

Example 8 — Multi-class: slice out one class (Case I)

Forecast: dog's TP is the (dog, dog) diagonal $=90$. Guess FP and FN before computing.

1. One-vs-rest for dog. Collapse the multi-class table back into our 2×2 grid by treating "dog" as positive and everything else as negative.

   • ${\text{TP}}_{\text{dog}}=90$ (the diagonal entry).
   • ${\text{FN}}_{\text{dog}}=8+2=10$ (dog row, off-diagonal — real dogs called something else).
   • ${\text{FP}}_{\text{dog}}=10+5=15$ (dog column, off-diagonal — others called dog). Why this step? Row-minus-diagonal = things of this class we missed; column-minus-diagonal = things we wrongly assigned here. This is the parent's per-class recipe, mapped onto the same top-row / left-column reading we have used all page.

2. Dog precision and recall. ${\text{Precision}}_{\text{dog}}=\frac{90}{90+15}=\frac{90}{105}\approx 0.857,{\text{Recall}}_{\text{dog}}=\frac{90}{90+10}=0.90$ Why this step? Same two ratios as always, now on the collapsed dog-vs-rest grid: precision over the dog column, recall over the dog row.

3. Micro-averaged precision — pool all classes' TP and FP first, then divide once. Total TP $=85+90+92=267$. Total off-diagonal (every misclassification is somebody's FP) $=10+5+8+2+3+5=33$. ${\text{Precision}}_{\text{micro}}=\frac{267}{267+33}=\frac{267}{300}=0.89$ Why this step? Micro pools counts first, so it weights by how many samples each class has. Macro-averaging (see Multi-class Classification Metrics) would instead average the three per-class precisions equally, ignoring class size.

Verify: dog recall $0.90$ = row $90/(90+10)$ ✓. Micro precision $0.89$ equals overall accuracy $\frac{267}{300}$ — in single-label multi-class every FP is another class's FN, so micro-precision = micro-recall = accuracy, a useful identity.

---

Example 9 — Exam twist: back-solve the matrix (Case J)

Forecast: you can get TP straight from recall. Try it before reading.

1. TP from recall. Recall $=\frac{\text{TP}}{200}=0.60$, so $\text{TP}=0.60\times 200=120$ Why this step? Recall's denominator is the known count of actual positives (the top row, $200$), so it pins TP immediately.

2. FN from the positive row. Actual positives $=$ TP $+$ FN, so $\text{FN}=200-120=80$ Why this step? The top row must sum to the given 200 actual positives, so whatever TP does not account for is FN.

3. FP from precision. Precision $=\frac{\text{TP}}{\text{TP}+\text{FP}}=0.75$, so $120=0.75(120+\text{FP})\Rightarrow 120+\text{FP}=\frac{120}{0.75}=160\Rightarrow \text{FP}=40$ Why this step? Precision links TP to the predicted-positive total (the left column); knowing TP lets us solve for FP algebraically.

4. TN from the negative row. Actual negatives $=$ FP $+$ TN $=800$, so $\text{TN}=800-40=760$ Why this step? The bottom row must sum to the given 800 actual negatives, so whatever is not a false alarm is a true negative.

5. Accuracy. $\text{Accuracy}=\frac{120+760}{1000}=0.88$ Why this step? With all four cells recovered, accuracy is the plain diagonal-over-total from its definition.

Verify: back-check both given metrics from the reconstructed cells — $\frac{120}{120+40}=0.75$ ✓ and $\frac{120}{120+80}=0.60$ ✓. Full matrix $[120,80;40,760]$ sums to $1000$ ✓. When they hand you metrics, invert them via their known denominators (see Model Selection Criteria).

---

Recall Self-test (reveal after guessing)

Predict-all-negative on imbalanced data: what is recall? ::: Exactly $0$ — you catch no positives, no matter how high accuracy climbs. Which metric has an undefined denominator when the model never predicts positive? ::: Precision (denominator $\text{TP}+\text{FP}=0$). As you lower the decision threshold, which of precision/recall is guaranteed non-decreasing? ::: Recall — admitting more positives can never lose a caught one. In single-label multi-class, micro-precision equals which familiar scalar? ::: Overall accuracy. Given precision, recall, and the actual-positive count, which cell do you solve for first? ::: TP, directly from recall $\times$ (actual positives).