Hallucination mitigation: Question bank — AI-ML deep dive

Before any trap, we pin down every term the reveals lean on. Nothing below is used before it is defined here.

Definition Core anchor terms — read these first

Factuality error ::: the output disagrees with the real world (regardless of any context given).
Faithfulness error ::: the output disagrees with the provided context/source, even if it happens to be true in the world.
Calibration ::: how well a model's stated confidence matches its actual chance of being right; a well-calibrated model that says "70%" is right about 70% of such times.
RAG (Retrieval-Augmented Generation) ::: retrieve relevant passages, paste them into the prompt, and instruct the model to answer only from them. Turns a world-truth question into a "match the given text" question. See RAG pipeline.
FActScore ::: a way to measure factuality. Break an answer into small independently-checkable atomic claims, mark each supported/unsupported against a trusted source, and take the fraction supported. If $N$ = total claims and $C$ = supported, faithfulness $= C / N$ and hallucination rate $= (N - C) / N$ .

Definition Temperature, threshold, coverage, risk — the numbers behind the traps

Temperature $T$ ::: a decoding dial. At each step the model has probabilities over next tokens; $T$ reshapes them. $T \to 0$ = always pick the single most-likely token (deterministic, "greedy"); large $T$ = flatter probabilities, more random low-probability picks.
Abstention threshold $τ$ ::: a confidence cut-off. The model reports a confidence score $s$ for its answer and only answers when $s \geq τ$ ; otherwise it abstains ("I don't know").
Coverage ::: the fraction of questions the model actually attempts, $Coverage = Pr (s \geq τ)$ . Higher $τ$ ⇒ fewer pass ⇒ coverage down.
Selective risk ::: the error rate among answered questions only, $Risk (τ) = \frac{# { answered and wrong }}{# { answered }} = Pr (wrong ∣ s \geq τ) .$ It is a conditional probability, so with zero answers it is $0/0$ = undefined.
RLHF (Reinforcement Learning from Human Feedback) ::: a training stage that tunes the model toward human-preferred answers. Because humans prefer helpful, confident replies, it can discourage honest "I don't know." See RLHF and alignment.

The picture below fixes the shape of coverage and risk as $τ$ moves — refer back to it for every $τ$ trap.

And this 2×2 grounds the two error axes the True/False items lean on — faithful-vs-factual are independent.

True or false — justify

An LLM is fundamentally a truth database that occasionally glitches.

False. It is a next-token predictor that outputs plausible continuations; truth is never directly optimised, so "glitch" mischaracterises the normal mechanism.

Perfect RAG retrieval guarantees a faithful answer.

False. Even with the right passage in context the model can misread, over-generalise, or ignore it — a faithfulness failure — which is why we still require citations and verification. See RAG pipeline.

Setting temperature

T

to 0 removes all hallucinations.

False. At

T = 0

decoding is greedy so it kills sampling-driven fabrications, but knowledge-gap hallucinations remain: the single most-likely sequence can itself be a confident invention (see the two-sources box above).

A hallucination and a factual error are exactly the same thing.

False. A hallucination can be a faithfulness error — contradicting the given context while still being true in the world — so the categories overlap but are not identical (see the 2×2 figure).

Raising the abstention threshold

τ

always lowers coverage.

True. A higher

τ

means fewer answers clear

s \geq τ

, so coverage

= Pr (s \geq τ)

can only stay equal or drop — trace the coverage curve leftward as

τ

rises in figure s01.

If an answer is 90% correct it should score 0.9 faithfulness under FActScore.

True only if claims are equally weighted: FActScore averages over atomic claims, so

C / N = 9/10 = 0.9

— the point is one lie doesn't condemn the whole answer.

Self-consistency helps because averaging text smooths out errors.

False. It works by voting over independently sampled answers: truths are stable attractors, fabrications scatter, so the majority tends to be the fact. You don't average text, you count agreement (see figure s03).

A well-calibrated model can still hallucinate.

True. Calibration only fixes the confidence it reports; a calibrated model can correctly say "I'm 40% sure" and still be wrong 60% of the time — calibration makes the uncertainty usable, not zero.

Chain-of-verification works even if the model checks its draft using the same reasoning that produced it.

False. The check must be independent; re-using the original reasoning just re-justifies the same error, so Chain-of-thought reasoning-style self-checks need fresh, separately-answered verification questions.

Faithfulness and factuality can be measured with a single accuracy number.

False. They fail against different references (context vs. world), so an answer can be faithful but false, or true but unfaithful — they need separate evaluation (the two independent axes of figure s02). See Evaluation metrics for LLMs.

Spot the error

"I gave the model the document, so any date it outputs must come from the document."

Error: grounding provides the facts but does not force their use; the model can inject a date from pretraining memory instead — a faithfulness violation. Require citations tied to the passage.

"The answer sounds confident and detailed, so it's reliable."

Error: fluency and confidence are the same for true and false LLM outputs; surface tone is not evidence of correctness because the model is often miscalibrated.

"Hallucination rate = 1 minus whole-answer accuracy."

Error: whole-answer scoring loses signal — one invented claim among nine correct ones would count the entire answer as wrong. Decompose into atomic claims first (that's what FActScore does).

"RLHF makes models more honest, so it reduces hallucination."

Error: RLHF and alignment often rewards being helpful and confident, which can penalise honest abstention and thereby increase confident guessing. Honesty must be trained/prompted for explicitly.

"Coverage of 100% with low risk is always the goal."

Error: forcing 100% coverage (set

τ

to its minimum) means answering even hopeless questions, pushing selective risk up. On high-stakes tasks the right goal is low risk at acceptable coverage, not full coverage.

"Self-consistency needs temperature 0 to be reliable."

Error: it needs diversity to reveal instability, so it requires nonzero

T

; at

T = 0

every sample is identical and voting tells you nothing.

Why questions

Why does forcing "answer only from the context" make the problem easier?

It converts an open-world factuality task (hard, unbounded) into a bounded faithfulness task that can be checked directly against the retrieved text and audited with citations.

Why do we break answers into atomic claims before measuring?

Because truth is a per-claim, not per-paragraph, property; decomposition (the core of FActScore) lets one wrong fact be isolated rather than condemning or excusing the whole answer.

Why does a smooth lie score as well as a clunky truth during training?

The training objective rewards likely continuations, and a fluent false statement can be just as probable as an awkward true one — likelihood and truth are decoupled.

Why can burying evidence in a long prompt still cause hallucination?

Because of the lost-in-the-middle effect: relevant context in the middle of a long input gets under-weighted, so the model falls back on its priors and may fabricate.

Why is citation-forcing a detection tool as much as a prevention tool?

Ungrounded claims show up as uncited, making invented content visible and filterable — so citations help you catch failures, not only reduce them.

Why does self-consistency work better on facts than on genuinely unknown answers?

Real facts act as attractors the sampling process keeps returning to, giving a strong majority; when the model truly doesn't know, samples scatter and no clean majority forms — which itself signals uncertainty (figure s03). See Model calibration.

Edge cases

What happens to selective risk when

τ

is set so high that the model answers zero questions?

Selective risk is undefined (

0/0

) because it is a conditional error rate and there are no answered questions in the denominator.

What does a hallucination look like when the retrieved passage is itself wrong?

The model can be perfectly faithful (matches the source) yet factually wrong — grounding propagated a bad fact, so faithfulness metrics look fine while factuality fails (the "faithful but false" cell of figure s02).

If every sampled answer disagrees (no majority), what should self-consistency conclude?

Not a confident answer — the scatter itself is a signal of low reliability, so this is a case to abstain or flag rather than pick a winner.

T = 0

with a knowledge gap, what is the model's behaviour?

It deterministically returns its single most-likely token sequence, which may be a confident fabrication — low randomness does not create knowledge.

If a model is perfectly calibrated but the task allows no abstention, can hallucination rate still be high?

Yes. Forced to answer (no

τ

cut-off available), it must guess on cases it "knows" are uncertain; calibration identifies the risk but can't reduce the wrong-answer count without an abstain option.

What is the limiting behaviour of coverage and risk as

τ \to 0

Coverage

\to 1

(it answers everything) and selective risk approaches the model's overall unconditional error rate, since no filtering by confidence occurs — the right-hand end of figure s01.

Recall One-line self-audit

Cover every answer above and re-derive the reason, not the verdict. If you can only recall "true/false" you have memorised noise — the justification is the transferable skill. See Prompt engineering for turning these principles into instructions.