2.6.8 · D5Model Evaluation & Selection
Question bank — Precision-recall tradeoff and curves
This is a D5 question bank for the parent topic. Before we let you loose on traps, we lock down the four words every question below leans on — so nothing appears unexplained.
Now answer each item out loud before revealing. The answer side always gives the reasoning, never a bare yes/no.
True or false — justify
Lowering the threshold can never decrease recall.
True — a lower only adds items to the positive set, so can only grow or stay equal; recall never drops.
Lowering the threshold can never decrease precision.
False — precision often falls because new positives include false alarms, but it can also rise if a newly-crossed item happens to be a true positive; the trend is downward, not a guarantee each step.
Precision and recall are always mathematically forced to move in opposite directions.
False — they tend to oppose across thresholds, but a genuinely better model can push both up at once; the "tradeoff" is a curve-shape observation, not a law forcing every point to trade one for one.
A PR curve is always monotonically decreasing from left to right.
False — real PR curves are jagged and non-monotonic; precision can jump up when the next item crossed is a true positive. Only the overall trend descends.
AUC-PR of 0.6 is good because it beats the 0.5 random line.
False — 0.5 is the random line for ROC, not PR. The PR no-skill baseline equals the positive-class fraction, which may be 0.02, making 0.6 excellent, or 0.7, making it mediocre.
A model with higher AUC-ROC always has higher AUC-PR.
False — on imbalanced data huge keeps FPR tiny so ROC looks great while precision (ignoring ) collapses; the two rankings can disagree. See 2.6.07-ROC-curves-and-AUC.
Precision depends on the number of true negatives.
False — contains no term, which is exactly why PR curves stay sensitive on imbalanced data where negatives dominate.
The point (recall = 1, precision = 1) is achievable only by a perfect classifier.
True — recall 1 means zero misses () and precision 1 means zero false alarms (); a model with both makes no mistakes at that threshold.
Setting always gives recall = 1.
True — every item scores , so all are predicted positive, , and recall ; precision then equals the positive-class fraction.
Spot the error
"Our fraud model has 0.08 precision but 0.80 recall on a 1% fraud set, and AUC-ROC is 0.9, so it's production-ready."
The error: ROC hides the flood of false alarms. Precision 0.08 means 92 of every 100 alerts are wrong — operationally unusable despite the pretty ROC; judge it on PR for imbalanced data.
"We compared AUC-PR = 0.15 to 0.5 and concluded the model is barely better than chance."
Wrong baseline. The PR chance line is the positive fraction; if fraud is 2%, baseline is 0.02, so 0.15 is a 7.5× lift, not "barely better."
"To get both precision and recall to 0.9, just pick the threshold where the PR curve crosses 0.9 on both axes."
A single threshold gives one point. If no point on the curve reaches (0.9, 0.9), no threshold can; you'd need a better model, not a cleverer .
"Averaging precision at 0.7 and recall at 0.7 gives model quality 0.7."
A plain average is meaningless because the two are at possibly different thresholds and weight differently; use the harmonic mean F1 or an if you must combine.
"AUC-PR near 1 means the model is well-calibrated."
AUC-PR measures ranking/separation quality across thresholds, not whether scores equal true probabilities. Calibration is a separate property — see 3.2.04-Decision-thresholds-and-calibration.
"We tuned the threshold on the test set to maximize F1, then reported that F1."
Tuning on the same data you report on leaks information and inflates the score; choose the threshold on a validation split.
Why questions
Why do PR curves beat ROC curves specifically on imbalanced data?
Because precision omits , it stays sensitive to false alarms even when negatives vastly outnumber positives, whereas ROC's FPR is diluted by the giant and looks deceptively good. See 2.6.12-Class-imbalance-handling.
Why is the PR no-skill baseline the positive-class fraction and not 0.5?
A random guesser's precision equals the chance a flagged item is truly positive, which is just the prevalence of positives — independent of threshold.
Why does the leftmost part of a PR curve (very low recall) often look noisy or jump around?
At high thresholds only a handful of items are predicted positive, so is tiny and precision swings wildly with each single correct or wrong call.
Why can't you improve precision by simply relabelling more items as negative wholesale?
Refusing to flag anything drives , collapsing recall; precision may look fine on the few remaining calls but you've stopped doing the job of finding positives.
Why does cost-sensitive learning change where you sit on the PR curve rather than the curve itself?
The curve is fixed by the model's scores; costs only shift the operating threshold to trade misses against false alarms — see 4.1.08-Cost-sensitive-learning.
Why is F1 sometimes preferred over reporting precision and recall separately?
F1's harmonic mean punishes a large gap between the two, so a model can't hide a terrible recall behind excellent precision — 2.6.09-F1-score-and-Fbeta-score.
Edge cases
What is precision when the model predicts zero positives ()?
Precision is — undefined. By convention it's often set to 1 (no false alarms) or excluded; recall is 0 since nothing was caught.
What happens to the PR curve if a dataset has no positive examples at all?
Recall's denominator , so recall is undefined and the PR curve cannot be drawn — the whole precision-recall framework needs at least one positive.
At just above the highest score, what are recall and precision?
No item crosses, so : recall is 0 (caught nothing) and precision is undefined/conventionally 1 — this is the top-left anchor of the curve.
If every item has the identical score, what does the PR curve degenerate to?
The model can only either flag all or none, so you get just two points; the curve becomes a single horizontal segment at precision = positive-fraction, showing no ranking ability.
For a perfectly-ranked model (all positives score above all negatives), what shape is the PR curve?
A right-angle: precision stays 1 as recall climbs to 1, then drops — because every early item flagged is a true positive until the positives run out. AUC-PR = 1.
Two thresholds give the same recall but different precision — is that possible?
Yes: crossing a false positive (raises , unchanged ) lowers precision while recall is untouched, producing a vertical drop on the PR curve.