Bayes' theorem — derivation and applications: Visual walkthrough

2.7.9 · D2 · Maths › Statistics & Probability — Intermediate › Bayes' theorem — derivation and applications

Yeh page parent note ki picture-companion hai. Agar koi symbol yahan anjaan lage, toh woh ek step pehle draw kiya gaya tha.

Step 1 — Poori duniya ek square hai jiska area 1 hai

KYA. Ek square draw karo jiska total area $1$ ke barabar ho. Har possible outcome iske andar kahin na kahin rehta hai. "Area" ka matlab poore page par "probability" hoga — bada patch matlab zyada likely cheez.

KYUN. Probability bas "sab outcomes mein se kya fraction?" hai. Area $1$ wala square us fraction ko kuch aisa bana deta hai jo tum literally dekh aur ruler se measure kar sako. Yeh wahi bookkeeping hai jaise Tree Diagrams mein hoti hai, bas ek aisi picture par flat kar di gayi hai jahan areas add up hote hain.

PICTURE. Neeche ke figure mein, poora peach square sample space hai. Humne abhi kuch nahi kaata — total area $= 1$ .

Figure — Bayes' theorem — derivation and applications

Step 2 — Square ko cause $A$ se kaato

KYA. Square ko ek vertical line se kaato. Line ke baayein sab kuch $A$ hai (cause hua); daayein sab kuch $\overset{ˉ}{A}$ hai ("not $A$ ", cause nahi hua).

KYUN. Hum pehle cause ke hisaab se split karte hain kyunki hum aksar uski base rate jaante hain. Left strip ki width exactly prior $P (A)$ hai — cause kitna likely hai kisi bhi evidence se pehle.

PICTURE. Magenta strip ki width $P (A)$ hai; baaki ki width $P (\overset{ˉ}{A}) = 1 - P (A)$ hai.

$magenta strip ki width P (A) + baaki ki width P (\overset{ˉ}{A}) = 1$

$P (A)$ — poore square ka woh fraction jo cause hai. Ek width hai.
$P (\overset{ˉ}{A}) = 1 - P (A)$ — bacha hua hissa, "poora square 1 hai" ki wajah se forced hai.

Step 3 — Har strip ke andar, evidence $B$ se kaato

KYA. Ab horizontally kaato, lekin har strip mein alag height par. $A$ strip ke andar, jo fraction evidence $B$ bhi dikhata hai woh $P (B ∣ A)$ hai. $\overset{ˉ}{A}$ strip ke andar, woh fraction $P (B ∣ \overset{ˉ}{A})$ hai.

KYUN. Yeh likelihood hai — "given ki main is strip mein hoon, $B$ part kitna tall hai?". Yeh poore square ka fraction nahi, balki ek strip ke andar ki height hai. $P(B\mid A)$ ko zor se padho jaise " $B$ ki probability, jab hum pehle se jaante hain ki hum $A$ ke andar hain." Yeh Conditional Probability ki definition hai jo visible ho gayi: ek chosen region ke andar ek proportion.

PICTURE. Do shaded bands — $A$ strip mein ek tall band (height $P (B ∣ A)$ ), ek $\overset{ˉ}{A}$ strip mein (height $P (B ∣ \overset{ˉ}{A})$ ).

$P (B ∣ A) = sirf A strip ke andar measure ki gayi shaded band ki height$

$P (B ∣ A)$ — magenta strip ke andar ek height hai. Iska denominator woh strip hai, square nahi.
$P (B ∣ \overset{ˉ}{A})$ — doosri strip ke andar wahi idea. Tests mein yeh false-positive rate hoti hai.

Step 4 — Chaar rectangles: joint probabilities AREAS hain

KYA. Dono cuts chaar rectangles chhodti hain. Kisi bhi rectangle ka area width $\times$ height hai. Top-left rectangle ka area $P (A) \times P (B ∣ A)$ hai — woh chance ki $A$ aur $B$ dono ho jaayein.

KYUN. Area $=$ width $\times$ height isliye humne square draw kiya. Strip ki width ko band ki height se multiply karna poore square ka ek genuine fraction deta hai — ek joint probability. Symbolic taur par:

$P (A \cap B) = strip width P (A) \times band height P (B ∣ A)$

PICTURE. Magenta corner rectangle highlight kiya gaya hai. Uski dono sides label ki gayi hain; uska area joint probability hai.

$P (A \cap B)$ — ek corner rectangle ka area (dono events true). Symbol $\cap$ ka matlab "aur" hai.
Equation bas "area = width × height" hai jo probability ke kapde pehne hue hai.

Step 5 — Wohi area, doosri taraf se kaata gaya

KYA. Yeh trick hai. Identical square ko pehle evidence $B$ se kaato (height $P (B)$ ki ek horizontal strip), phir cause se. $A$ -and- $B$ corner wahi physical rectangle hai — uska area sirf isliye nahi badal sakta kyunki humne uski sides alag order mein measure ki.

KYUN. Ek area compute karne ke do honest tarike agree karne chahiye. Toh:

$cause-first kaata P (A) P (B ∣ A) = P (A \cap B) = evidence-first kaata P (B) P (A ∣ B)$

Yahi Bayes ka poora engine hai: ek area, do decompositions.

PICTURE. Left panel: cause-first kaato. Right panel: evidence-first kaato. Same green rectangle, alag labelled sides.

$P (B)$ — width... maafi, height evidence strip ki jab hum $B$ pehle kaatein: poore square ka woh fraction jo evidence dikhata hai.
$P (A ∣ B)$ — posterior, woh cheez jo hum actually chahte hain: evidence diya, us strip ka kitna hissa cause hai?

Step 6 — Posterior ko isolate karne ke liye divide karo

KYA. Step 5 ki equalities ki chain lo aur dono sides ko $P (B)$ se divide karo.

KYUN. Hum $P (A ∣ B)$ ko akela left par chahte hain. $P (B)$ se divide karna baaki sab kuch hata deta hai. Term by term:

$P (A ∣ B) = \frac{P ( B ∣ A ) likelihood: band height P ( A ) prior: strip width}{evidence: total B area P ( B )}$

PICTURE. Posterior literally "green rectangle area ÷ full evidence strip area" hai. Figure green corner ko poori $B$ strip ke andar dikhata hai — us strip ka woh fraction jo bhi $A$ hai.

Step 7 — $P (B)$ kahan se aata hai? Evidence strip pieces se banti hai

KYA. Poori evidence strip ek rectangle nahi hai — yeh $A$ strip ka $B$ -part plus $\overset{ˉ}{A}$ strip ka $B$ -part hai.

KYUN. Evidence $B$ dikhane wala har point theek ek cause-strip mein rehta hai (woh overlap nahi karte aur square ko bharte hain — ek partition). Toh hum dono green areas jodte hain. Yeh exactly Law of Total Probability hai:

$P (B) = green area in A strip P (B ∣ A) P (A) + green area in \overset{ˉ}{A} strip P (B ∣ \overset{ˉ}{A}) P (\overset{ˉ}{A})$

PICTURE. Dono green pieces glow karte hain; right par ek bracket dikhata hai ki woh total evidence height $P (B)$ tak stack karte hain.

In dono areas ko jodhna hi denominator hai. Isliye parent note $P (B) = \sum_{i} P (B ∣ A_{i}) P (A_{i})$ likhta hai — yahan $n = 2$ hai.
Agar tum ek piece bhuul jaao, toh sab causes ke fractions $1$ mein nahi judenge — ek classic mistake.

Step 8 — Degenerate aur edge cases (kabhi reader ko giraao mat)

KYA AUR KYUN. Real problems mein weird inputs aate hain. Picture har case mein yeh karti hai:

Worked example Square par case-by-case

$P (B) = 0$ (evidence impossible). Evidence strip ki height zero hai — fraction "green ÷ 0" undefined hai. Bayes simply apply nahi hota; tum kisi aisi cheez par update nahi kar sakte jo kabhi hoti hi nahi.
$P (A) = 0$ (cause a priori impossible). Magenta strip ki width zero hai, toh uska green corner zero area ka hai. Phir $P (A ∣ B) = 0$ : koi evidence aisi cause ko wapas nahi la sakta jise tumne impossible maana tha. Prior ek gate hai.
$P (A) = 1$ (cause certain). Poora square $A$ strip hai; $\overset{ˉ}{A}$ strip gayab ho jaati hai. Phir $P (B) = P (B ∣ A)$ aur $P (A ∣ B) = 1$ — evidence total certainty ko hila nahi sakta.
$B$ aur $A$ independent — dekho Independent Events. Band height dono strips mein same hai: $P (B ∣ A) = P (B ∣ \overset{ˉ}{A}) = P (B)$ . Phir $P (A ∣ B) = P (A)$ : evidence tumhe kuch nahi sikhata, posterior prior ke barabar hai.
Test perfectly specific ( $P (B ∣ \overset{ˉ}{A}) = 0$ ). Right strip mein koi green nahi. Har green point $A$ mein hai, toh $P (A ∣ B) = 1$ : ek positive ek guaranteed cause hai.

Figure zero-width, zero-height, aur independent cases ke liye teen shrunken squares dikhata hai taaki tum fraction ko collapse hote dekh sako.

Ek-picture summary

Sab kuch ek square par. Poora derivation yeh hai: unit square ko cause se kaato, phir evidence se; ek corner joint hai; us corner ko poori evidence strip se divide karo; aur strip khud apne pieces ka sum hai. Woh ek sentence neeche ka figure hai.

$P (A ∣ B) = \frac{P ( B ∣ A ) height P ( A ) width}{green in A P ( B ∣ A ) P ( A ) + green in A ˉ P ( B ∣ A ˉ ) P ( A ˉ )}$

Recall Feynman: poora walkthrough plain words mein

Duniya ko ek square rug socho jiska total area one hai. Ek vertical line khiincho: left part "cause true hai" aur woh part kitna wide hai yeh batata hai ki cause kitna common hai kisi bhi clue se pehle — yahi tumhara prior hai. Ab har half ke andar, woh strip colour karo jahan clue appear hota hai. Left half mein coloured patch "cause AND clue" hai. Yeh punchline hai: tum us exact same patch ko horizontally pehle kaatke colour kar sakte the — same patch, uski sides describe karne ke do tarike. Un dono descriptions ko equal karo, total coloured height se divide karo, aur Bayes' theorem nikal aata hai. "Clue diya, kya cause true hai?" ka jawab bas yeh hai: SAARI coloured area ka kitna slice cause half mein baitha hai? Agar cause impossibly rare tha (zero width), uska coloured patch zero hai, aur koi clue use nahi bacha sakta. Agar clue dono halves mein identical lagta hai, clue useless hai aur tumhara belief nahi badlta. Yahi Bayes hai — ek rug par areas compare karne ke siwa kuch nahi.

Recall

Area picture mein, Bayes' theorem ka numerator kya represent karta hai? ::: Us single rectangle ka area jahan cause aur evidence dono hote hain: $P (B ∣ A) P (A)$ . Picture mein, denominator $P (B)$ kya represent karta hai? ::: Poori evidence strip ka area — har cause-strip ke coloured pieces ka sum. $P (A) = 0$ kyun force karta hai $P (A ∣ B) = 0$ ? ::: Zero-width cause-strip ka ek zero-area corner hota hai, toh koi evidence use positive posterior nahi de sakta — prior ek gate hai. Picture mein posterior prior ke barabar kab hota hai? ::: Jab evidence band dono strips mein equal height ka ho (independence): evidence koi information nahi carry karta.

Connections

Conditional Probability — har "strip ke andar height" ek conditional probability hai.
Law of Total Probability — Step 7: evidence strip pieces ke sum ke roop mein.
Independent Events — woh edge case jahan posterior prior ke barabar hota hai.
Tree Diagrams — areas ki jagah branch form mein wahi bookkeeping.
Naive Bayes Classifier — yahi square, kai features ke liye repeat kiya gaya.
Prior and Posterior Distributions — strip widths ka continuous version.