4.4.15
Intuition 80/20 core ideaEk LLM ek next-token predictor hai, na ki ek truth database . Yeh hamesha kuch na kuch plausible-sounding output karta hai, chahe uske paas koi grounding ho ya na ho. Ek hallucination woh confident output hai jo fluent toh hai lekin factually galat ya unsupported hai . Mitigation ka matlab hai model ko force karna ki woh (a) sirf wahi kahe jo support kar sake , (b) answers ko retrieved evidence mein ground kare , aur (c) jab unsure ho toh abstain kare . Aap hallucinations ko delete nahi kar sakte; aap sirf unki rate aur cost kam kar sakte ho .
Ek LLM output jo ya toh (i) duniya ke saath unfaithful hai (factually galat → factuality error) ya (ii) diye gaye source/context ke saath (given text ko contradict kare → faithfulness error). Key point: yeh ek sahi answer ki tarah hi same high fluency aur confidence ke saath generate hoti hai, isliye surface cues se pata nahi chalta.
Do flavors jo aapko alag karne chahiye:
Type
Fails against
Example
Main fix
Factuality real-world truth
"The capital of Australia is Sydney"
Retrieval (RAG), better data
Faithfulness the given context
Doc kehta hai 2019; model kehta hai 2021
Grounding + citations
Intuition Root causes (steel-manned)
Objective mismatch — training likely continuations ko reward karta hai, true ones ko nahi. Ek smooth lie utni hi achhi score karti hai jitni ek clunky truth.Knowledge gaps — woh fact training data mein rare/absent tha, toh model ek plausible fill-in interpolate kar deta hai.No calibrated "I don't know" — RLHF aksar model ko helpful aur confident banne ki training deta hai, jo honest abstention ko penalize karta hai.Decoding randomness — high temperature low-probability, kabhi kabhi false, tokens sample karta hai.Context overload / lost-in-the-middle — relevant evidence ignore ho jaata hai agar woh beech mein daba hua ho.
Hum model ko claims ke classifier ke roop mein treat karte hain {supported, unsupported} mein.
Intuition Mental model: teen wallsWall 1 — Grounding (generation se pehle): model ko facts do (RAG). Sabse bada single win.
Wall 2 — Constrained generation: isse cite karwao, step-by-step reason karwao, self-check karwao.
Wall 3 — Verification (generation ke baad): claims check karo, retry karo, ya abstain karo.
Retrieved passages ko prompt mein inject karo aur instruct karo: "Answer sirf context use karke do; agar present nahi hai, toh kaho ki pata nahi." Yeh ek factuality task ko ek faithfulness task mein convert karta hai, jo enforce aur audit karna kaafi aasaan hai (citations ke through). Dekho RAG pipeline .
[source] tags require karo. Ungrounded claims visibly uncited ho jaati hain → filter karna easy.
T → 0
k majority answer rakho. Hallucinations aksar samples mein unstable hoti hain, truths stable hoti hain.
Model draft karta hai → verification questions generate karta hai → unhe independently answer karta hai → revise karta hai. Independence isse rok deti hai ki woh sirf apni galti ko re-justify na kare.
Model ko train/prompt karo ki jab evidence missing ho toh "I don't know" output kare; upar wale risk–coverage curve se threshold τ
Worked example FActScore-style hallucination rate compute karnaAnswer: "Marie Curie ne do Nobel Prizes jeete (Physics 1903 , Chemistry 1911 ) aur polonium aur radium discover kiye University of Vienna mein."
Atomic claims: {do Nobels ✓, Physics 1903 ✓, Chemistry 1911 ✓, polonium discover kiya ✓, radium discover kiya ✓, University of Vienna mein ✗ (woh Paris tha)}.
Yeh step kyun? Hum independently checkable units mein todh dete hain taaki ek galat fact poore answer ko condemn na kare.
N = 6 , C = 5 ⇒ HallucinationRate = 6 6 − 5 = 0.167.
Worked example Abstention threshold choose karna5 answered questions ke liye confidences aur correctness:
( s , correct ) = ( 0.9 , ✓ ) , ( 0.8 , ✓ ) , ( 0.6 , × ) , ( 0.55 , ✓ ) , ( 0.4 , × ) τ = 0.5 = 1/4 = 0.25 = 4/5 = 0.8 τ = 0.7 = 0/2 = 0 = 2/5 = 0.4 Yeh step kyun? Yeh trade dikhata hai: tighter τ
Worked example Self-consistency in actionQ: "The Origin of Species kisne likhi?" 5 sample karo: Darwin, Darwin, Darwin, Wallace, Darwin.
Majority = Darwin (4/5). Akela "Wallace" ek hallucination hai jo voting se filter ho gayi.
Yeh step kyun? Sahi facts woh attractors hain jinpe model baar baar jaata hai; fabrications scatter ho jaati hain.
Common mistake "RAG hallucinations eliminate kar deta hai."Kyun sahi lagta hai: aapne isko facts de diye, toh zaroor use karega. Reality: model phir bhi context ko ignore, misread, ya over-generalize kar sakta hai (faithfulness failure), ya retriever ne galat passages fetch kiye hon. Fix: citations require karo, claims ko retrieved text ke against verify karo, aur faithfulness alag se evaluate karo.
Common mistake "Confident answer matlab correct answer."Kyun sahi lagta hai: humans mein confidence knowledge ke saath correlate karta hai. Reality: LLMs aksar miscalibrated hote hain — poori fabrications par fluent aur confident. Fix: tone nahi, calibrated uncertainty use karo (self-consistency spread, verifier scores).
Common mistake "Bas temperature = 0 set karo aur safe ho."Kyun sahi lagta hai: kam randomness = kam cheezon ko banana. Reality: T = 0 sampling hallucinations hatata hai lekin knowledge-gap hallucinations nahi — yeh confidently single most likely galat answer dega. Fix: low T
Common mistake "Poore answer par hallucination right/wrong ke roop mein measure karo."Kyun sahi lagta hai: simple hai. Reality: signal lose ho jaata hai — mostly-true answer ek jhooth ke saath garbage jaisa score karta hai. Fix: atomic claims mein decompose karo (FActScore).
Recall Quick self-test (chhupaao, phir jawab do)
Factuality aur faithfulness hallucination mein kya difference hai?
LLMs mein high confidence correct hone ki guarantee kyun nahi deta?
Abstention threshold τ
Answers ko atomic claims mein kyun decompose karte hain?
Recall Feynman: ek 12-saal ke bacche ko samjhaoEk bahut tez storyteller ki imagine karo jo kabhi stuck nahi dikhna chahta. Kuch bhi pucho aur woh turant ek smooth answer deta hai — jab woh actually jaanta bhi nahi, toh woh sirf kuch aisa bana deta hai jo sahi lagta hai . Woh bana hua hissa ek "hallucination" hai. Isko rokne ke liye hum teen cheezein karte hain: (1) pehle isko answer book de do (retrieval) aur kaho "sirf yahi use karo"; (2) isse apne sources dikhwao taaki hum invented bits pakad sakein; (3) isko "I'm not sure" kehne do guess karne ki jagah. Hum usi question ko kai baar bhi poochte hain — sahi answer same rehta hai, bana hua answer baar baar badalta hai, toh hum fake pakad lete hain.
G-C-A-V — "Grounded Cats Always Verify"
G round (RAG) · C ite sources · A bstain when unsure · V erify claims (self-check / self-consistency).
LLM hallucination kya hoti hai? Ek fluent, confident output jo factually galat hai ya given context se unsupported hai, kyunki model true tokens ki jagah likely tokens predict karta hai.
Factuality vs faithfulness hallucination? Factuality real-world truth ko contradict karta hai; faithfulness provided source/context ko contradict karta hai.
N atomic claims ke saath C supported ke liye hallucination rate ka formula? (N − C)/N.
Self-consistency hallucinations kyun reduce karta hai? Sahi facts samples mein stable hoti hain jabki fabrications vary karti hain, toh majority voting unstable galat answers filter kar deta hai.
Abstention threshold τ badhane ka effect? Coverage kam hoti hai (kum jawab diye) aur selective risk kam hoti hai (jawab diye gaye mein kum errors).
RAG hallucinations poori tarah kyun eliminate nahi karta? Model context ko ignore/misread kar sakta hai (faithfulness failure) ya retriever galat passages fetch kar sakta hai.
Temperature=0 akela insufficient kyun hai? Yeh sampling-based errors hatata hai lekin knowledge-gap errors nahi — yeh confidently single most likely galat answer return karta hai.
Chain-of-Verification (CoVe) ki core idea? Draft → verification questions generate karo → unhe independently answer karo → revise karo, taaki model apni galti ko re-justify na kare.
Scoring ke liye answers ko atomic claims mein kyun decompose karte hain? Taaki ek false claim true ones mein chup na jaaye; aapko ek granular faithfulness fraction milta hai (FActScore).
Selective risk ki definition? Error rate sirf un questions par calculate kiya gaya jinhe model ne answer karna choose kiya (confidence ≥ τ).
RAG pipeline — primary grounding mechanismPrompt engineering — "sirf context se answer do / I don't know kaho"RLHF and alignment — models over-confident kyun ho jaate hain aur abstention train kaise kareinModel calibration — raw confidence ko trustworthy uncertainty mein kaise badleinChain-of-thought reasoning — reasoning + verification stepsEvaluation metrics for LLMs — FActScore, risk–coverage curves
Factuality error vs world
Faithfulness error vs context
Objective mismatch, gaps, no IDK, decoding
HallucinationRate = N-C over N
Abstention above threshold tau
