Visual walkthrough — Bayesian statistics — prior, likelihood, posterior (intro)

4.9.24 · D2 · Maths › Probability Theory & Statistics › Bayesian statistics — prior, likelihood, posterior (intro)

Shuru karne ke liye sirf ek idea chahiye: probability is area. Aur kuch nahi.

---

Step 1 — Poori duniya area $1$ ka ek square hai

KYA HAI. Ek square banao. Uska total area $1$ hai. Har possible outcome uske andar kahin na kahin rehta hai. Square ke andar ek region ek event hai — koi cheez jo ho sakti hai — aur uski probability bas kitna square woh cover karti hai yahi hai.

KYUN. Humein ek aisi picture chahiye jo "probability" shabd ko kisi aisi cheez mein convert kare jise tum ruler se measure kar sako. Area bilkul wahi karta hai: area $\ge 0$ hamesha hota hai, aur poora square $1$ tak add hota hai, jo un do rules se match karta hai jo har probability follow karti hai.

PICTURE. Square ke andar do overlapping blobs hain. Lavender blob event $A$ hai; coral blob event $B$ hai.

Abhi tak kuch bhi abstract nahi hai — $P(A)$ literally paint coverage hai.

---

Step 2 — "Given $B$" ka matlab: baki square ko throw away karo

KYA HAI. Maano koi tumhe bata de ki "$B$ definitely hua." Ab coral blob $B$ tumhari bilkul nayi duniya hai. Hum zoom in karte hain taaki $B$ frame ko fill kare, aur poochte hain: is nayi duniya ke andar, kitna $A$ bhi hai?

KYUN. Conditioning multiplication nahi hai ya koi naya axiom — yeh re-scaling hai. Jab hum jaante hain ki $B$ true hai, toh square ke bahar ke hisse impossible ho gaye, isliye unhe count nahi kar sakte. Humein $A$ ko $B$ ko new "total" maan kar re-measure karna hoga.

PICTURE. Left frame mein purana square hai; right frame mein wahi $B$ poori duniya ban ke blow up ho gayi hai. Overlap $A\cap B$ hi $A$ ka ek surviving piece hai.

Dekho Conditional Probability — yeh fraction iska poora matlab hai, draw karke.

---

Step 3 — Overlap ko dono taraf se measure kiya ja sakta hai

KYA HAI. Wahi overlap patch $A\cap B$ ko do tarike se build kiya ja sakta hai:

• $B$ ki duniya mein shuru karo, uska $A$-fraction lo: $P(A\mid B)P(B)$;
• $A$ ki duniya mein shuru karo, uska $B$-fraction lo: $P(B\mid A)P(A)$.

KYUN. Overlap paint ka ek physical patch hai. Usse pata nahi ki hum "pehle" kis blob mein gaye. Toh dono recipes ko same area dena hi hoga. Yeh symmetry poore theorem ka seed hai.

PICTURE. Identical purple overlap tak do arrows pahunchte hain — ek pehle $B$ slice karta hai, ek pehle $A$ slice karta hai — aur dono same square pe land karte hain.

---

Step 4 — Ek baar divide karo aur Bayes nikalta hai

KYA HAI. Step 3 ke do outer pieces lo aur dono ko $P(B)$ se divide karo: $P(A\mid B)P(B)=P(B\mid A)P(A)⟹P(A\mid B)=\frac{P(B\mid A)P(A)}{P(B)}.$

KYUN. Hum chahte hain $P(A\mid B)$ — woh cheez jo hum directly nahi dekh sakte (jaise "kya main beemaar hun, test dekh ke?"). Hamare paas hai $P(B\mid A)$ — woh cheez jo labs measure karti hain (jaise "kya test fire karta hai, beemar hone par?"). $P(B)$ se divide karna conditioning bar ko flip karta hai aur jo hum chahte hain usse isolate karta hai.

PICTURE. "Flip the bar" label wala ek arrow $P(B\mid A)$ (measure karna aasaan) ko $P(A\mid B)$ (jo humein chahiye) mein turn karta hai, saath mein $P(A)/P(B)$ correction bhi chali aati hai.

---

Step 5 — Statistics ke liye blobs rename karo

KYA HAI. Maano $A=\theta$ (hypothesis / parameter — coin ka bias, beemar hona) aur $B=D$ (data jo humne actually observe kiya). Same square, naye labels.

KYUN. Statistics mein hum $\theta$ nahi dekhte; hum $D$ dekhte hain. Toh hum un ingredients se $P(\theta \mid D)$ chahte hain jo hum actually pa sakte hain.

PICTURE. Wahi identical two-blob square, ab lavender blob $\theta$ label se aur coral blob $D$ label se, aur har area pe uska statistics name tag kiya hua.

---

Step 6 — $P(D)$ kahan se aata hai? Data blob ko chop karo

KYA HAI. Denominator $P(D)$ coral blob ka total area hai. Data blob ko us piece mein split karo jo $\theta$ ke andar hai aur us piece mein jo "not-$\theta$" ($\bar{\theta }$) ke andar hai, phir add karo: $P(D)=\underset{\text{data via }\theta }{\underbrace{P(D\mid \theta )P(\theta )}}+\underset{\text{data via not-}\theta }{\underbrace{P(D\mid \bar{\theta })P(\bar{\theta })}}.$

KYUN. Data kisi bhi hypothesis ke through aa sakta hai. $D$ ka sahi size paane ke liye humein un sab tareekon par sweep karna hoga jis se yeh ho sakta tha — yahi Law of Total Probability hai. Kai hypotheses ke saath yeh ek sum ${\sum }_{\theta }$ ban jaata hai (ya continuous $\theta$ ke liye integral $\int$ hota hai).

PICTURE. Coral $D$ blob ko $\theta$ blob ki boundary do shaded strips mein slice karti hai; unke areas $P(D)$ tak add hote hain.

---

Step 7 — Degenerate cases (reader ko kabhi wall se mat takrao)

KYA HAI. Har woh tarika check karo jis se picture break ho sakti hai.

KYUN. Koi formula jis par tum trust karo usse apni extremes mein survive karna chahiye, sirf friendly middle mein nahi.

PICTURE. Chaar mini-squares: (a) koi overlap nahi, (b) $\theta$ $D$ ko nigal leta hai, (c) $P(\theta )=0$, (d) $P(D)=0$.

---

Ek-picture summary

Sab ek saath: ek unit square mein do blobs. Overlap numerator hai $P(D\mid \theta )P(\theta )$; poora coral blob denominator hai $P(D)$; unka ratio — overlap over data-world — posterior hai $P(\theta \mid D)$. Bayes bas "overlap, data world ka ek slice ke roop mein measure kiya hua" hai.

Recall Feynman retelling — poora walkthrough simple words mein

Duniya ko area $1$ ka ek square samjho: jo bhi ho sakta tha sab andar fit hai. Do paint blobs overlap karte hain — "mera guess sahi hai" (lavender) aur "yeh data maine dekha" (coral). Update karne ke liye, main coral blob ke bahar ki sab cheez bhool jaata hun, kyunki data sach mein hua — coral meri nayi duniya hai. Ab main poochta hun: is coral duniya ka kitna hissa lavender bhi hai? Woh fraction mera naya belief hai, posterior. Clever trick yeh hai ki overlap patch ko do tarike se measure kiya ja sakta hai — kisi bhi blob se shuru karke — aur un dono ko barabar set karke, phir coral area se divide karke, easy lab number ("test tab fire karta hai jab tum beemaar ho") ko woh number mein flip kar deta hai jo main actually chahta hun ("main beemaar hun kyunki test fire hua"). Neeche ka hissa, poora coral area, bas un sab tareekon ka total hai jis se data show ho sakta tha, jode hue. Aur warnings: koi overlap nahi matlab guess dead hai; jo guess maine zero chance diya tha woh dead rehta hai chahe kuch bhi dekh lun; aur main kabhi bhi aisi data pe condition nahi kar sakta jo impossible thi shuru se hi.

Recall Quick self-test

Hum $P(D)$ se divide kyun karte hain na ki $P(\theta )$ se? ::: Kyunki "given $D$" $D$ ko hamari nayi duniya banata hai; hum us duniya ke size se re-scale karte hain jis pe condition kiya, jo $D$ hai. Picture mein, Bayes' theorem ka numerator kaunsa region hai? ::: Overlap $A\cap B$, yaani $P(D\mid \theta )P(\theta )$. Agar $P(\theta )=0$ hai, toh kisi bhi data ke liye posterior kya hai? ::: Zero — ek ruled-out prior ko kabhi revive nahi kiya ja sakta. Agar $P(D)=0$ ho toh kya problem aati hai? ::: Zero se division; tum impossible data pe condition nahi kar sakte.

Related builds: Binomial Distribution aur Beta Distribution coin example ke liye shapes dete hain; Maximum Likelihood Estimation sirf likelihood rakhta hai (flat prior); Naive Bayes Classifier spam filters ke liye yeh square millions baar run karta hai.