Hallucination mitigation: Visual walkthrough — AI-ML deep dive

We assume you know nothing except: an LLM answers questions, and sometimes it is wrong. Everything else — confidence, threshold, risk, coverage — we build here. See the parent topic for the big picture and Model calibration for where "confidence" comes from.

Step 1 — A question, an answer, and a "how sure am I?" number

WHAT. Picture one question going into the model. Two things come out: an answer, and a confidence score — a single number $s$ between $0$ and $1$ saying how sure the model is. $s = 1$ means "totally certain", $s = 0$ means "pure guess".

WHY. We cannot control whether the model is wrong, but we can read its confidence and decide whether to trust it. To make any decision, we first need a number to threshold. That number is $s$ .

PICTURE. Look at the figure: the question enters the black box, and out comes a dot on a horizontal ruler running from $0$ (left) to $1$ (right). The red dot is this question's confidence $s$ . That ruler is the whole stage on which the rest of the derivation plays out.

Step 2 — Many questions become dots on the same ruler

WHAT. Now send in a whole batch of questions. Each produces its own confidence $s_{i}$ (the subscript $i$ just names the question: question 1, question 2, …). We drop all of them onto the same $0$ -to- $1$ ruler.

WHY. One dot tells us nothing about a policy. A cloud of dots lets us ask: "if I only trust dots to the right of some line, how many do I keep, and how many of those are actually wrong?" That question is the entire game.

PICTURE. In the figure, each dot is one question. We color a dot red if the answer was actually wrong, black if correct. Notice the wrong (red) dots cluster toward the low-confidence left — that clustering is the only reason thresholding helps at all. If wrong answers were spread evenly, no threshold could save us.

Step 3 — Drop the threshold line $τ$

WHAT. Draw a vertical line at position $τ$ (Greek letter "tau", just a name for "the cutoff"). Rule: answer only if $s_{i} \geq τ$ , otherwise say "I don't know." (abstain).

WHY. This is the one knob we control. We can't fix the dots' positions, but we can slide this line. Every mitigation decision in the parent note — "abstain when unsure" — is literally choosing where this line stands.

PICTURE. Everything to the right of the red line is answered; everything to the left is abstained. Watch the red line: as it slides right, it swallows more territory into "abstain", and — crucially — it eats the leftmost wrong dots first.

Step 4 — Count two things: Coverage and Risk

WHAT. With the line placed, count only the answered dots (right of the line) and define two fractions.

$Coverage (τ) = \frac{# { i : s _{i} \geq τ }}{N}$

Term by term: the numerator $# {i : s_{i} \geq τ}$ = how many dots are right of the line (the $# {\dots}$ means "count of"); the denominator $N$ = total questions. So Coverage = fraction of questions we dared to answer.

$Risk (τ) = \frac{# { i : s _{i} \geq τ and answer wrong }}{# { i : s _{i} \geq τ }}$

Term by term: numerator = wrong dots that are right of the line (red dots we still answered); denominator = all dots right of the line (everything we answered). So Risk = error rate among only the answered questions.

WHY. These are the two numbers the parent formula uses. Coverage measures helpfulness (did we attempt it?), Risk measures danger (of the ones we attempted, how many blew up?). Notice Risk divides by the answered count, not $N$ — that is what makes it "selective": we don't get blamed for questions we honestly skipped.

PICTURE. The figure boxes the answered region and labels the two counts inside it: total answered (the box) and wrong-answered (the red dots in the box). Risk is the red fraction of that box; Coverage is the box's size relative to the whole ruler.

Step 5 — Slide the line and watch both numbers move

WHAT. Take the same dots and move $τ$ from far left to far right, recording $(Coverage, Risk)$ at each stop.

WHY. A single $τ$ is one data point. The relationship — how Risk changes as Coverage changes — is the object we actually want, because it tells us the price of safety.

PICTURE. Three snapshots side by side:

$τ$ far left (line near $0$ ): we answer everything. Coverage $\approx 1$ , but every wrong dot is included → Risk is at its maximum.
$τ$ middle: we've cut the leftmost red dots. Coverage drops, Risk drops.
$τ$ far right (line near $1$ ): we answer only the most confident few. Coverage tiny, Risk usually near zero.

The red line marches rightward across the three panels; the wrong-dot count inside the box shrinks faster than the total — that's why Risk falls.

Intuition Why the curve is (mostly) monotone — Risk can only fall as

$τ$ rises Slide $τ$ right by a tiny amount. The only dots that leave the answered set are the ones sitting just above the old line — the lowest-confidence answered dots. If confidence is informative (lower $s$ = more likely wrong, which is the whole reason we bothered), those departing dots are wrong at least as often as the average of the answered set. Removing a group whose error rate is $\geq$ the current average can only pull the average down or leave it flat — never up. So under this "informativeness" assumption, as $τ$ rises Coverage falls and Risk is non-increasing. Read on the curve (Coverage horizontal): moving leftward (less coverage) the curve trends downward (less risk). It is not guaranteed monotone for a badly-calibrated model — a stray high-confidence wrong dot can bump Risk up locally — but the overall trend holds whenever confidence carries real signal.

Step 6 — Plot Risk against Coverage: the curve

WHAT. Put Coverage on the horizontal axis and Risk on the vertical axis. Each threshold $τ$ gives one point $(Coverage (τ), Risk (τ))$ . Connect them.

WHY. This single curve answers the design question directly: "If my job requires Risk below (say) $5%$ , how many questions can I still answer?" Read across at Risk $= 0.05$ , drop down — that's your maximum Coverage. No formula needed once you have the curve.

PICTURE — first fix the axes in your head. Coverage is the horizontal axis, running $0$ (left) to $1$ (right) — how many questions we answered. Risk is the vertical axis, running $0$ (bottom) to $1$ (top) — how often we were wrong among those. So "bottom-left" = few answers, almost never wrong; "top-right" = all answers, wrong as often as the raw model. Now read the figure: the red curve sweeps from the bottom-left (tiny coverage, near-zero risk — the ultra-cautious corner) up toward the right end at Coverage $= 1$ . That right endpoint sits at whatever the model's raw overall error rate is — it is the highest risk on the curve, but it is not necessarily $1$ ; if the model is right most of the time, this corner sits low. The curve generally rises left-to-right (more coverage buys more risk); a good model hugs the bottom, a badly calibrated one bulges up early.

Formula One number for the whole curve — AURC

To compare two models by a single score, measure the Area Under the Risk–Coverage curve (AURC) — literally the area sitting beneath the red curve. With the curve sampled at coverage points $c_{1} < c_{2} < \dots < c_{M}$ (each from some $τ$ ) with risks $r_{1}, \dots, r_{M}$ , approximate the area by the trapezoid rule (summing thin trapezoids under the curve): $AURC \approx \sum_{k = 1}^{M - 1} avg height of strip \frac{r _{k} + r _{k + 1}}{2} \cdot strip width (c_{k + 1} - c_{k})$ Term by term: each strip's height is the average of the two neighbouring risks, its width is the gap in coverage; multiply and sum. Smaller AURC = a better safety/coverage trade-off (the curve stays low across all coverages). A perfect ranker (all wrong answers below all correct ones) drives AURC toward the tiny area of an all-correct prefix; a useless ranker gives a flat curve whose AURC equals the raw error rate. See Evaluation metrics for LLMs for the exact conventions.

Step 7 — The edge cases (never leave the reader guessing)

WHAT. Check the two extreme thresholds and the pathological model, so no scenario surprises you.

WHY. A derivation you trust must survive its own boundaries. If the formula misbehaved at $τ = 0$ or $τ = 1$ , we'd have a bug.

PICTURE. The figure marks three special spots on the curve:

$τ = 0$ (answer everything). Coverage $= 1$ . Risk $=$ the model's raw overall error rate. This is the far-right endpoint — the "no abstention" baseline. Nothing is filtered.
$τ = 1$ (answer only perfectly-certain items). Under the rule $s_{i} \geq τ$ , we now keep only answers with $s_{i} = 1$ exactly. Two sub-cases: (a) if no answer has $s_{i} = 1$ , the answered set is empty, Coverage $= 0$ , and Risk is genuinely undefined ( $0/0$ ). (b) if some answers pile up exactly at $s_{i} = 1$ (a "mass at 1"), those few are answered, so Coverage $> 0$ but tiny, and Risk is computed from that handful — meaning one lucky correct item gives Risk $= 0$ , one unlucky wrong item gives Risk $= 1$ . Either way the far-right tip is statistically unstable (few or zero samples), so we never trust it. Note: as $τ$ approaches $1$ from below, Coverage tends toward this small mass, not necessarily to $0$ .
An uncalibrated, uninformative model where confidence is pure noise, unrelated to correctness: the red dots' colors are spread evenly across the ruler, so every threshold keeps the same red fraction → the curve is a flat horizontal line. (This is precisely not calibrated — its confidence carries no information.) Thresholding buys you nothing. This is the worst case and the diagnostic: if your risk–coverage curve is flat, your confidence signal is useless.

Step 8 — Reading the curve to pick $τ$ (the payoff)

WHAT. Redo the parent note's worked example on the curve. Recall the five answered questions: $(s, correct) = (0.9, ✓), (0.8, ✓), (0.6, \times), (0.55, ✓), (0.4, \times) .$

WHY. To prove the picture and the arithmetic agree.

PICTURE. Two labelled points on the curve (remember: Coverage horizontal $0 \to 1$ , Risk vertical $0 \to 1$ ):

$τ = 0.5$ : answered $= 4$ (drop the $0.4$ ). Wrong among them $= 1$ (the $0.6$ ). $Risk = 1/4 = 0.25$ , $Coverage = 4/5 = 0.8$ .
$τ = 0.7$ : answered $= 2$ (the $0.9, 0.8$ ), both right. $Risk = 0/2 = 0$ , $Coverage = 2/5 = 0.4$ .

Sliding $τ$ from $0.5$ to $0.7$ walks you down-and-left along the red curve: Risk $0.25 \to 0$ , Coverage $0.8 \to 0.4$ . You traded half your coverage to erase all your risk. Whether that trade is worth it depends on the cost of a wrong answer — high in medicine, low in brainstorming.

The one-picture summary

Everything above collapses into one diagram: the dot-ruler on top (where confidence lives and where the red threshold line slides) feeding the risk–coverage curve below (where each line position becomes one point). Move the line → move along the curve → choose your safety.

Recall Feynman retelling — explain the whole walkthrough to a friend

Imagine every answer the model gives comes with a "how sure am I?" sticker, a number from 0 to 1. Line all the answers up on a ruler by that sticker. Now draw a line and say: "I'll only answer the ones to the right of the line; the rest I'll skip." If you push the line right, you skip more questions (that's less coverage) but the ones you keep are the confident ones, which are usually right (that's less risk). Plot "how much I skip" against "how often I'm still wrong" and you get a curve. Reading it is the whole trick: decide the biggest mistake-rate you can stomach, and the curve tells you how many questions you're still allowed to answer. Two warnings: don't slam the line all the way right (you answer so few that the risk number becomes random noise), and if the curve is dead flat, your confidence sticker is worthless — the model has no idea when it's guessing.

Active recall

Recall Self-test

Why do wrong answers get removed faster than correct ones as $τ$ rises? ::: Wrong answers cluster at low confidence, so a rightward threshold eats them first. What happens to Risk as $τ \to 1$ , and why is it untrustworthy there? ::: The answered count shrinks to a handful (or zero), so Risk is estimated from almost no samples — unstable, or undefined ( $0/0$ ) when nothing has $s = 1$ . What does a flat risk–coverage curve tell you about the model? ::: Its confidence score carries no information — thresholding gains nothing. Under the "informative confidence" assumption, what happens to Risk as $τ$ rises? ::: It is non-increasing — you always remove dots at least as error-prone as the current answered average. At $τ = 0$ , what does Risk equal? ::: The model's raw overall error rate (nothing is filtered, so Coverage $= 1$ and every wrong answer is counted).

Related: Model calibration · RAG pipeline · Chain-of-thought reasoning · RLHF and alignment · Prompt engineering