Worked examples — Precision-recall tradeoff and curves

This is a Deep Dive drill sheet for the parent topic on the precision-recall tradeoff. The parent showed you what precision and recall are and why they oppose each other. Here we do something different: we walk into every corner of the problem space and work each one out by hand — including the weird, degenerate, "wait, what does that even mean?" cases that break naive formulas.

Before we start, two symbols we will lean on constantly. If either feels shaky, re-read the confusion-matrix note.

Notice precision never touches TN and recall never touches TN either. That single fact is the seed of half the surprises below.

---

The scenario matrix

Every precision-recall problem you will ever meet is one of these cells. Each worked example below is tagged with the cell(s) it covers.

#	Cell class	What makes it tricky	Covered by
A	Ordinary threshold sweep	move $\tau$, recompute both metrics	Ex 1
B	Threshold at extremes	$\tau =0$ (flag everything), $\tau =1^{+}$ (flag nothing)	Ex 2
C	Zero-denominator / degenerate	$TP+FP=0$ (precision is $0/0$); no positives exist	Ex 3
D	Severe class imbalance	huge TN hides a terrible precision	Ex 4
E	Baseline / lift	is a number actually good vs. random?	Ex 5
F	AUC-PR by hand	trapezoid integration over the curve	Ex 6
G	Real-world word problem	translate a story into the four counts	Ex 7
H	Exam twist: work backwards	given precision & recall, recover the counts	Ex 8
I	Comparing two models on the same point	same recall, tie-break on precision	Ex 9

---

Example 1 — Cell A: an ordinary threshold sweep

Forecast: a high threshold like 0.75 keeps only the most confident flags. Guess: precision high, recall low.

1. Apply the rule "flag positive if score $\ge \tau$." At $\tau =0.75$ the flagged set is $\{e1,e2\}$. Why this step? The threshold is the classifier here — it's the single knob that turns scores into yes/no decisions (see decision thresholds).
2. Sort each flagged email into a bucket. e1 = spam ✔ (TP), e2 = spam ✔ (TP). So $TP=2$, $FP=0$. Why this step? Precision and recall are built purely from these buckets — nothing else matters.
3. Count the misses. True-spam emails not flagged: e4, e6 → $FN=2$. Why this step? Recall's denominator needs all real positives, flagged or not.
4. Plug in. $\text{Precision}=\frac{2}{2+0}=1.00$, $\text{Recall}=\frac{2}{2+2}=0.50$. Why this step? We substitute the counts from steps 2–3 into the two definitions — this is the moment the raw buckets turn into the two numbers we actually report. We do both at once because they share the same $TP$, so a single count-up feeds both.

Verify: $TP+FN=2+2=4$ = total true spam ✔. Precision $1.00$ means zero false alarms — consistent with $FP=0$. Forecast confirmed: high precision, half recall.

---

Example 2 — Cell B: the two extreme thresholds

Forecast: at $\tau =0$ you flag everything → you catch all spam (recall = 1) but drown in ham. At $\tau >1$ you flag nothing.

1. $\tau =0$: flag every email. $TP=4$ (all spam), $FP=4$ (all ham), $FN=0$. Why this step? $\tau =0$ is the "shout at everyone" limit — the far-right end of every PR curve.
2. Compute. $\text{Precision}=\frac{4}{4+4}=0.50$, $\text{Recall}=\frac{4}{4+0}=1.00$. Why this step? Recall is always 1 here — you can never miss a positive if you flag all. Precision collapses to the positive fraction $4/8=0.50$. Remember that number; it's the baseline of Cell E.
3. $\tau =1.01$: flag nothing. $TP=0$, $FP=0$, $FN=4$. Why this step? This is the "never shout" limit — the far-left end of the curve.
4. Compute recall. $\text{Recall}=\frac{0}{0+4}=0.00$. Precision is $\frac{0}{0+0}$ — undefined ($0/0$). We handle that in Example 3. Why this step? We substitute $TP=0,FN=4$ into recall to show that flagging nothing gives zero recall (every positive is a miss), and we deliberately try precision too so the $0/0$ crash surfaces here rather than surprising us later — it motivates Example 3.

Verify: $TP+FP$ at $\tau =0$ is $8$ = total emails ✔ (we flagged all). The recall values $1.00$ and $0.00$ bracket the whole curve — every other threshold lives between them. ✔

---

Example 3 — Cell C: the degenerate $0/0$ and the empty-positive case

Forecast: both are $\frac{0}{0}$. Math says "undefined"; software has to choose a convention. Guess: they choose whatever keeps curves sane.

1. (a) Write the fraction. $\text{Precision}=\frac{TP}{TP+FP}=\frac{0}{0}$. Why this step? You must see the zero denominator to know a raw formula will crash.
2. Resolve it by convention. The standard (scikit-learn) rule: precision $:=0$ when nothing is flagged. Rationale: "you made no correct positive calls, so your positive-call quality is 0." Why this step? Without a convention the PR curve would have a hole at recall $=0$; the convention plugs it so the curve starts cleanly at $(r=0,p=1)$ by extrapolation or $p=0$ by definition depending on library — always check yours.
3. (b) Write recall's fraction. $\text{Recall}=\frac{0}{0+0}=\frac{0}{0}$. Why this step? No positives exist to catch, so "fraction caught" is genuinely meaningless.
4. Resolve it. Convention: recall $:=0$ (or reported as undefined and the sample excluded). Practically, never evaluate recall on a set with zero positives — the metric can't be defined.

Verify: sanity — if you cannot be penalised for a miss (no positives to miss), recall carries no information, so "undefined / 0" is the honest answer. The key lesson, not a number: always check whether a denominator can be zero before trusting a metric. ✔

---

Example 4 — Cell D: severe imbalance hides a terrible precision

Forecast: with 9 950 negatives, false alarms barely dent the ROC's FPR, but precision — which ignores those negatives — should expose the problem.

1. Fill the four counts. Flagged 250, of which $TP=40$ ⇒ $FP=250-40=210$. Real cancer $=50$, caught $40$ ⇒ $FN=10$. Then $TN=10000-250-10=9740$. Why this step? Every metric below is just a ratio of these four numbers.
2. Precision. $\frac{40}{40+210}=\frac{40}{250}=0.16$. Why this step? Precision only sees $TP$ and $FP$ — the 9 740 correct passes are invisible to it.
3. Recall. $\frac{40}{40+10}=\frac{40}{50}=0.80$. Why this step? We caught 80% of cancers — sounds great in isolation.
4. ROC FPR for contrast. $\text{FPR}=\frac{FP}{FP+TN}=\frac{210}{210+9740}=\frac{210}{9950}\approx 0.0211$. Why this step? The huge $TN$ in the denominator crushes FPR to ~2%, making ROC look excellent while precision is a dismal 16%.

Verify: counts sum to the total: $40+210+10+9740=10000$ ✔. The clash — recall 0.80 & FPR 0.02 (rosy) vs. precision 0.16 (grim) — is exactly why imbalanced problems demand PR curves, not ROC.

---

Example 5 — Cell E: baseline and lift

Forecast: the PR baseline is not 0.5; it's the positive fraction, which is tiny here. So 0.15 might actually be strong.

1. Compute the positive fraction. $\pi =\frac{50}{10000}=0.005$. Why this step? A random classifier's expected precision equals the prevalence of positives — no matter the threshold — so $\pi$ is the baseline AUC-PR.
2. Compare against it. Model $0.15$ vs. baseline $0.005$. Why this step? 0.15 looks small only against the wrong yardstick (0.5). Against 0.005 it's enormous.
3. Lift = model / baseline. $\frac{0.15}{0.005}=30$. Why this step? Lift converts "how good" into "how many times better than guessing" — the honest headline number.

Verify: baseline via a second route — a random classifier flagging fraction $f$ of items catches $f\cdot 50$ positives and $f\cdot 9950$ negatives, so precision $=\frac{50f}{50f+9950f}=\frac{50}{10000}=0.005$, independent of $f$ ✔. The colleague was wrong: 30× lift is excellent. Compare baseline logic with cost-sensitive learning where the "good enough" bar is set by dollars, not lift.

---

Example 6 — Cell F: AUC-PR by hand (trapezoid rule)

Forecast: we're integrating precision over recall. Points slope downward, so the area should land somewhere between 0.4 and 1.0 — probably mid-0.6s.

Reading the figure: the four black dots are our known PR points P1–P4, joined by the bold black curve. The dashed black verticals slice the region under the curve into three strips; each strip is a trapezoid (its two vertical sides are the precisions $p_i$ and $p_{i+1}$, its width is $\Delta r$). The red shaded region is the AUC-PR we are about to compute — steps 3–5 below are literally adding up those three red strips, and step 6 gives the total the red label announces.

1. Sort by increasing recall. Already sorted (0.00 < 0.40 < 0.70 < 1.00). Why this step? The trapezoid rule marches left→right along the x-axis; out-of-order points would give nonsense (even negative) areas.
2. Why trapezoids at all? We only know precision at a few recall values, so we connect adjacent points with straight lines and sum the strip areas. Each strip is a trapezoid — two parallel vertical sides $p_i,p_{i+1}$ and width $\Delta r$.
3. Strip P1→P2. width $0.40$, avg height $\frac{1.00+0.90}{2}=0.95$ → area $0.95\times 0.40=0.380$. Why this step? A trapezoid's area = (average of its two parallel sides) × width; here the sides are the two precisions and the width is the recall gap — this is the leftmost red strip in the figure.
4. Strip P2→P3. width $0.30$, avg height $\frac{0.90+0.60}{2}=0.75$ → area $0.75\times 0.30=0.225$. Why this step? Same trapezoid formula on the next adjacent pair; we move rightward one strip at a time so no area is double-counted or skipped.
5. Strip P3→P4. width $0.30$, avg height $\frac{0.60+0.40}{2}=0.50$ → area $0.50\times 0.30=0.150$. Why this step? The final red strip; because precision has dropped to 0.40 here, this strip contributes the least — you can see it is the shortest region in the figure.
6. Sum. $0.380+0.225+0.150=0.755$. Why this step? The whole area under a piecewise-linear curve is exactly the sum of its strips, so adding the three trapezoids gives the total AUC-PR — the value printed in red on the figure.

Verify: widths add to the full recall span $0.40+0.30+0.30=1.00$ ✔. The result $0.755$ sits inside $[0.40,1.00]$ as forecast, and since precision starts high and only dips late, a value in the mid-0.7s is sensible. ✔ This same number feeds into F1 / Fβ discussions when you pick an operating point.

---

Example 7 — Cell G: real-world word problem

Forecast: 18 of 32 alerts were real → precision a bit above half. 18 of 20 floods warned → recall high.

1. Extract the counts from the story. "32 alerts, 18 real" ⇒ $TP=18$, $FP=32-18=14$. "20 real floods, 18 warned" ⇒ $FN=20-18=2$. Why this step? Word problems are just the four counts in disguise — always translate first.
2. Precision. $\frac{18}{18+14}=\frac{18}{32}=0.5625$. Why this step? Precision answers the resident's question "when the siren sounds, how often is it real?" — here 56%, so nearly half the sirens are false alarms.
3. Recall. $\frac{18}{18+2}=\frac{18}{20}=0.90$. Why this step? Recall answers "of floods that actually happened, how many were warned?" — 90%, so the city misses 2 in 20.

Verify: $TP+FP=32$ = total alerts ✔; $TP+FN=20$ = total floods ✔. Trade-off in words: to push recall to 100% (never miss a flood) the city would lower the alert bar and precision would fall further — the classic tension from the parent note.

---

Example 8 — Cell H: work backwards (exam twist)

Forecast: both formulas share $TP$; if we know $TP$ we can algebraically invert each ratio.

1. Invert precision for $FP$. $0.75=\frac{30}{30+FP}\Rightarrow 30+FP=\frac{30}{0.75}=40\Rightarrow FP=10$. Why this step? Precision is the only equation containing $FP$, so it's our lever for it.
2. Invert recall for $FN$. $0.60=\frac{30}{30+FN}\Rightarrow 30+FN=\frac{30}{0.60}=50\Rightarrow FN=20$. Why this step? Likewise recall is the only equation containing $FN$.
3. Total real positives. $TP+FN=30+20=50$. Why this step? The denominator of recall is the count of all positives in the data.

Verify: plug back. Precision $=\frac{30}{30+10}=\frac{30}{40}=0.75$ ✔. Recall $=\frac{30}{30+20}=\frac{30}{50}=0.60$ ✔. Both reproduced from the recovered counts, so $FP=10$, $FN=20$ and 50 true positives are all correct — the example closes cleanly.

---

Example 9 — Cell I: tie-break two models at the same recall

Forecast: same recall means same catches; the winner is whoever raised fewer false alarms → higher precision. Fewer total flags (B) should win.

1. Recover each model's $FP$. $FP=(\text{total flags})-TP$. Model A: $160-80=80$. Model B: $100-80=20$. Why this step? Total positive predictions $=TP+FP$; subtract the known $TP$ to isolate false alarms.
2. Precision A. $\frac{80}{80+80}=\frac{80}{160}=0.50$. Why this step? We substitute Model A's counts into the precision definition to price its false alarms.
3. Precision B. $\frac{80}{80+20}=\frac{80}{100}=0.80$. Why this step? At equal recall, precision is the deciding metric — it measures the false-alarm cost of getting the same catches.

Verify: both recalls equal $\frac{80}{100}=0.80$ ✔ (confirming a fair comparison). Model B wins, precision 0.80 vs 0.50 — it achieves the identical catch rate with far less collateral. This is exactly the tie-break a PR curve visualises: at a fixed x (recall), the higher curve (precision) is the better model.

---

Recall Self-check

At $\tau =0$ (flag everything), what is recall always equal to? ::: Exactly $1.0$ — you cannot miss a positive if you flag every example. Why can precision be a $0/0$ undefined value? ::: When the model flags nothing, $TP+FP=0$, so the fraction has a zero denominator; libraries define it (usually as 0) by convention. What is the no-skill baseline for AUC-PR? ::: The positive-class fraction (prevalence), not 0.5. Two models tie on recall — what breaks the tie? ::: Precision: whoever raised fewer false alarms for the same catches.