Bayesian statistics — prior, likelihood, posterior (intro): Worked examples

4.9.24 · D3 · Maths › Probability Theory & Statistics › Bayesian statistics — prior, likelihood, posterior (intro)

Yeh page parent topic ka drill-ground hai. Har worked example ek scenario matrix ke cell ke saath tagged hai, taaki padhne ke baad tumne har tarah ka case dekha ho jo Bayes' theorem de sakta hai: discrete aur continuous, rare aur common events, strong aur weak priors, zero-data aur infinite-data limits, aur kuch exam-style twists.

Agar koi symbol aaye jo tumhe pehchaan na aaye, toh hum use yahan build karte hain — koi assumed notation nahi.

Scenario Matrix

Har Bayes problem is grid ke ek cell mein rehta hai. Columns hain unknown kis tarah ka hai; rows hain wo tricky feature jo answer ka flavor badal deti hai.

Cell	Jo feature test ho rahi hai	Example jo ise cover karta hai
A	Discrete unknown, ordinary numbers	Ex 1 — disease test
B	Rare event (tiny prior ek achhe test ko bhi dabaa deta hai)	Ex 2 — bahut rare disease
C	Sequential updating (posterior agla prior ban jaata hai)	Ex 3 — do tests ek ke baad ek
D	Continuous unknown, flat prior	Ex 4 — coin, uniform prior
E	Continuous unknown, informative prior	Ex 5 — coin, Beta(a,b) prior
F	Zero-data / degenerate limit	Ex 6 — kuch flip nahi, sab heads
G	Large-data limit (data prior ko duba deta hai)	Ex 7 — 1000 mein 700 heads
H	Real-world word problem with a twist	Ex 8 — spam filter
I	Exam twist: prior odds & Bayes factor	Ex 9 — odds form

Neeche har numeric answer machine-checked hai verify block mein.

Shuru karne se pehle, do words jo hum baar baar use karenge:

Cell A — Discrete unknown, ordinary numbers

Worked example Ex 1 — Disease test, 1% prevalence

Disease $1%$ logon ko affect karti hai. Test $99%$ sensitive hai ( $P (+ ∣ sick) = 0.99$ ) aur $95%$ specific ( $P (- ∣ healthy) = 0.95$ , toh false-positive rate $0.05$ ). Tum positive test karte ho — toh yahan data hai $D = +$ . $P (sick ∣ +)$ find karo.

Forecast: Abhi ek gut guess likho. Zyaatar log kehte hain "lagbhag $99%$ ." Is baat ko yaad rakho.

Step 1 — do hypotheses aur prior list karo. $θ \in {sick, healthy}$ , $P (sick) = 0.01$ , $P (healthy) = 0.99$ . Yeh step kyun? Bayes ko prior weights ke saath exhaustive list of hypotheses chahiye; yahan sirf do hain.

Step 2 — har hypothesis ke under data ( $+$ ) ki likelihood likho. $P (+ ∣ sick) = 0.99$ , $P (+ ∣ healthy) = 0.05$ . Yeh step kyun? Likelihood batati hai ki har hypothesis ne jo humne dekha use kitne achhe se explain kiya — Conditional Probability dekho.

Step 3 — evidence via Law of Total Probability. $P (+) = 0.99 (0.01) + 0.05 (0.99) = 0.0099 + 0.0495 = 0.0594$ Yeh step kyun? $P (+)$ ko saare tareekon par average karna padega jisse $+$ aa sake.

Step 4 — posterior. $P (sick ∣ +) = \frac{0.99 \times 0.01}{0.0594} = \frac{0.0099}{0.0594} \approx 0.1667$ Yeh step kyun? Yeh Bayes' theorem khud hai: "ek akela path jo sick aur $+$ dono hai" ko "saare paths to $+$ " se divide karna conditioning ko $P (+ ∣ sick)$ (jo hum jaante the) se $P (sick ∣ +)$ (jo hum chahte hain) mein flip karta hai.

Verify: Complement se cross-check: $P (healthy ∣ +) = \frac{0.0495}{0.0594} = 0.8333$ , aur $0.1667 + 0.8333 = 1$ . ✓ Do posterior probabilities certainty ko partition karte hain.

Sirf ~17% — $1%$ base rate " $99%$ test" ke gut guess ko crush kar deta hai. Simple baat hai, healthy logon ki itni badi population false positives generate karne ke liye kaafi hai.

Cell B — Rare event (tiny prior ek achhe test ko bhi dabaa deta hai)

Worked example Ex 2 — Same test, prevalence

$0.1%$ kar di Ab disease sirf $0.1%$ logon ko affect karti hai ( $P (sick) = 0.001$ ). Test dono taraf $99%$ accurate hai. Tum positive test karte ho (data $D = +$ ). $P (sick ∣ +)$ find karo.

Forecast: Padhne se pehle guess karo. Kya yeh Ex 1 ke $17%$ se zyada ya kam hoga?

Step 1 — priors. $P (sick) = 0.001$ , $P (healthy) = 0.999$ . Yeh step kyun? Prevalence hi prior hai; ise Ex 1 se $10 \times$ chota karna hi ek alag change hai, isliye pehle batana padega taaki iska effect alag se dekh sakein.

Step 2 — likelihoods. $P (+ ∣ sick) = 0.99$ , $P (+ ∣ healthy) = 0.01$ . Yeh step kyun? Yeh describe karte hain ki har hypothesis $+$ ko kaise explain karti hai; " $99%$ accurate dono taraf" sensitivity ko $0.99$ aur false-positive rate ko $0.01$ fix karta hai.

Step 3 — evidence. $P (+) = 0.99 (0.001) + 0.01 (0.999) = 0.00099 + 0.00999 = 0.01098$ Yeh step kyun? Phir se Law of Total Probability — normalise karne se pehle hum dono hypotheses par $+$ ki total chance chahiye.

Step 4 — posterior. $P (sick ∣ +) = \frac{0.00099}{0.01098} \approx 0.0902$ Yeh step kyun? Bayes' theorem: true-positive path sabhi positive paths se divide hota hai $P (+ ∣ sick)$ ko us answer mein flip karne ke liye jo hum chahte hain, $P (sick ∣ +)$ .

Verify: Sanity — prevalence ko aur zyada halve karne par answer $17%$ se $9%$ tak gaya, bilkul "rarer disease $\Rightarrow$ zyada false positives" trend. ✓

Figure — Bayesian statistics — prior, likelihood, posterior (intro)

Cell C — Sequential updating (posterior agla prior ban jaata hai)

Worked example Ex 3 — Do independent tests, dono positive

Prevalence $1%$ , har test $99%$ sensitive aur $95%$ specific (jaise Ex 1 mein). Tum test do baar, independently, lete ho aur dono positive aate hain (data $D = + +$ ). $P (sick ∣ + +)$ find karo.

Forecast: Ex 1 mein ek positive se $17%$ mila tha, doosra positive kya karega?

Step 1 — Ex 1 ka posterior naya prior banao. Pehle $+$ ke baad, $P (sick) = 0.1667$ . Yeh step kyun? Bayesian update recursive hai: kal ka posterior aaj ka prior hai. Independent data ke liye order matter nahi karta.

Step 2 — doosre test ke liye same likelihoods. $P (+ ∣ sick) = 0.99$ , $P (+ ∣ healthy) = 0.05$ . Yeh step kyun? Test nahi badla aur doosra result pehle se independent hai, isliye iske likelihoods pehle test jaise hi hain — machine behetr ya kharab nahi hui.

Step 3 — 2nd test ke liye evidence. $P (+_{2}) = 0.99 (0.1667) + 0.05 (0.8333) = 0.16500 + 0.04167 = 0.20667$ Yeh step kyun? Hum doosre update ko updated prior ( $0.1667$ / $0.8333$ ) ke against normalise karte hain, original $0.01$ ke against nahi — doosre $+$ ki total probability ko wahi reflect karna chahiye jo hum ab sochte hain.

Step 4 — dono ke baad posterior. $P (sick ∣ + +) = \frac{0.99 \times 0.1667}{0.20667} \approx 0.7984$ Yeh step kyun? Phir se Bayes' theorem, post-first-test belief feed karke; yeh dono positives ke baad belief deliver karta hai.

Verify: Ek hi shot mein karo — do positives ke liye likelihoods multiply karo: $\frac{0.9 9 ^{2} ( 0.01 )}{0.9 9 ^{2} ( 0.01 ) + 0.0 5 ^{2} ( 0.99 )} = \frac{0.009801}{0.009801 + 0.0024750} \approx 0.7984$ Same number ✓ — confirm karta hai ki sequential updating $=$ batch updating.

Ek positive ne hum $17%$ par chhoda; doosra independent positive hum ~80% tak le jaata hai. Evidence accumulate karna, honestly kiya gaya.

Cell D — Continuous unknown, flat prior

Worked example Ex 4 — Coin bias, 10 mein 7 heads, uniform prior

Unknown bias $θ \in [0, 1]$ . Data hai $D =$ " $10$ tosses mein $7$ heads". Prior $P (θ) = 1$ . Posterior aur uska mean find karo.

Forecast: Kya posterior mean exactly $0.7$ hoga, ya kahin pull hoga?

Step 1 — likelihood Binomial hai. $P (D ∣ θ) = (7 10) θ^{7} (1 - θ)^{3}$ Yeh step kyun? Har head ek factor $θ$ contribute karta hai, har tail ek factor $(1 - θ)$ ; binomial coefficient sirf orderings count karta hai aur $θ$ mein constant hai.

Step 2 — proportional posterior. $P (θ ∣ D) \propto θ^{7} (1 - θ)^{3} \cdot 1$ Yeh step kyun? Posterior $\propto$ likelihood $\times$ prior; flat prior $1$ hai aur constant $(7 10)$ normalization mein absorb ho jaata hai.

Step 3 — family pehchano. $θ^{8 - 1} (1 - θ)^{4 - 1} \Rightarrow θ ∣ D \sim Beta (8, 4)$ . Yeh step kyun? Beta, Binomial ke liye conjugate hai: Beta $(1, 1) + 7$ heads $+ 3$ tails $=$ Beta $(8, 4)$ .

Step 4 — posterior mean. $\frac{a}{a + b} = \frac{8}{12} = 0.6 \overline{6}$ .

Verify: Mean $0.667$ , prior mean $0.5$ aur data fraction $0.7$ ke beech hai — prior estimate ko thoda $0.5$ ki taraf shrink karta hai. ✓

Cell E — Continuous unknown, informative prior

Worked example Ex 5 — Same 7/10 data, lekin Beta(2,2) prior ke saath

Tumhara pehle se vishwas hai ki coin roughly fair hai: prior $= Beta (2, 2)$ ( $0.5$ par ek gentle hump). Data phir se hai $D =$ " $10$ mein $7$ heads". Posterior aur uska mean find karo.

Forecast: Ex 4 ke $0.667$ se compare karte hue, kya mean $0.5$ ki taraf move karega ya door?

Step 1 — prior as clay. $Beta (2, 2) \propto θ^{1} (1 - θ)^{1}$ . Step 2 — likelihood se multiply karo. $P (θ ∣ D) \propto θ^{7} (1 - θ)^{3} \cdot θ^{1} (1 - θ)^{1} = θ^{8} (1 - θ)^{4}$ Yeh step kyun? Phir conjugacy: bas counts add karo. Beta $(2, 2)$ ke saath $7$ heads, $3$ tails $\to$ Beta $(2 + 7, 2 + 3)$ . Step 3 — posterior. $Beta (9, 5)$ . Step 4 — mean. $\frac{9}{9 + 5} = \frac{9}{14} \approx 0.6429$ .

Verify: $0.6429 < 0.6667$ : "fair" prior ne flat prior se zyada estimate ko further $0.5$ ki taraf kheencha. ✓ Neeche ke figure mein plum posterior orange wale se left mein hai — tum literally extra pull dekh sakte ho.

Cell F — Zero-data aur degenerate limits

Worked example Ex 6 — Agar tum (a) kuch observe nahi karte, (b) sab heads aate hain?

Flat prior Beta(1,1) wala coin. (a) Tum zero baar flip karte ho (data $D = \emptyset$ ). Posterior kya hai? (b) Tum 3 baar flip karte ho, 3 heads aate hain (data $D =$ " $3$ heads"). Posterior mean kya hai, aur kya $θ = 1$ certain hai?

Forecast: (b) ke liye, kya 3-for-3 prove karta hai ki coin hamesha heads aata hai?

Part (a) — zero data. Step: Koi data nahi, toh likelihood har $θ$ ke liye constant ( $1$ ) hai, isliye posterior $\propto$ prior. Kyun? Koi data nahi $\Rightarrow$ update karne ke liye kuch nahi $\Rightarrow$ posterior $=$ prior $=$ Beta $(1, 1)$ , mean $0.5$ . Verify (a): Posterior exactly prior ke barabar. ✓ Degenerate case handle ho gaya: Bayes kabhi bhi information invent nahi karta.

Part (b) — sab heads. Step 1: Likelihood $\propto θ^{3} (1 - θ)^{0} = θ^{3}$ . Step 2: Posterior $\propto θ^{3} \Rightarrow Beta (4, 1)$ . Step 3: Mean $= \frac{4}{4 + 1} = 0.8$ . $1$ kyun nahi? Ek proper prior (Beta $(1, 1)$ ) ek patla slice belief rakhta hai ki $θ < 1$ ; posterior edge par peak karta hai lekin uska mean $0.8$ par rehta hai, $1$ nahi. Bayes finite data se certain hone se mana karta hai.

Verify (b): Mean $= 0.8$ aur $P (θ = 1 ∣ D) = 0$ kyunki continuous density single point par zero mass daalti hai. ✓ Maximum Likelihood Estimation se compare karo, jo $\hat{θ} = 1$ return karega — do philosophies ke beech ek cautionary contrast.

Cell G — Large-data limit (data prior ko duba deta hai)

Worked example Ex 7 — 1000 mein 700 heads, Beta(2,2) prior

Strongly-fair prior Beta $(2, 2)$ , lekin ab data ka pahaad: $D =$ "1000 tosses mein 700 heads". Posterior mean?

Forecast: Ex 5 mein same prior ne $0.7$ ko $0.643$ tak kheencha tha. $1000$ tosses ke saath, kitna pull bachega?

Step 1 — counts add karo. Beta $(2 + 700, 2 + 300) = Beta (702, 302)$ . Yeh step kyun? Conjugacy updating ko pure bookkeeping banata hai: heads $a$ mein add karo, tails $b$ mein. Yahan do prior "pseudo-counts" $700$ aur $300$ real counts ke saamne tiny hain — preview ki prior needle ko almost move nahi karta.

Step 2 — mean. $\frac{702}{1004} \approx 0.6992$ . Yeh step kyun? Posterior mean $\frac{a}{a + b}$ hamaara single-number summary hai; ise compute karne se hum measure kar sakte hain ki prior ka kitna pull data ke flood se bacha.

Verify: $0.6992$ raw data fraction $0.700$ se ek baal under hai — prior ka influence lagbhag khatam ho gaya. ✓ Jaise jaise data badhta hai, Bayesian estimate MLE / frequentist answer ki taraf converge hota hai. Prior sirf tab matter karta hai jab data scarce ho.

Cell H — Real-world word problem (twist)

Worked example Ex 8 — Spam filter with two clue words

Ek naive-Bayes spam filter. Prior: $30%$ email spam hai. Spam mein, word "free" prob $0.6$ se aata hai; ham mein, prob $0.1$ . Spam mein, "meeting" prob $0.05$ se aata hai; ham mein, prob $0.4$ . Ek email mein "free" aur "meeting" dono hain — yeh pair hamaara data $D$ hai. Kya yeh spam hai?

Forecast: Ek word spam chillata hai, doosra ham chillata hai. Kaun jeetega?

Step 1 — maano ki do words class diye jaane par conditionally independent hain (woh "naive" wala hissa). Yeh step kyun? Isse hum per-word likelihoods ko multiply kar sakte hain impossible joint tables estimate karne ki jagah. Step 2 — pair ki likelihood. $P (words ∣ spam) = 0.6 \times 0.05 = 0.03, P (words ∣ ham) = 0.1 \times 0.4 = 0.04$ Step 3 — priors se weight karo. $spam score = 0.03 \times 0.30 = 0.009, ham score = 0.04 \times 0.70 = 0.028$ Step 4 — normalise karo. $P (spam ∣ words) = \frac{0.009}{0.009 + 0.028} = \frac{0.009}{0.037} \approx 0.2432$

Verify: $P (ham ∣ words) = \frac{0.028}{0.037} \approx 0.7568$ aur dono ka sum $1$ hai. ✓ "free" ke bawajood, email ~76% ham hai — "meeting" aur ham-heavy prior jeet jaate hain.

Cell I — Exam twist: odds form & Bayes factor

Worked example Ex 9 — Ex 1 ko odds form mein redo karo

Prevalence $1%$ , sensitivity $0.99$ , false-positive $0.05$ , observed $D = +$ . Odds se $P (sick ∣ +)$ find karo.

Forecast: Same $17%$ milna chahiye jaise Ex 1 mein — method par ek consistency check.

Step 1 — prior odds. $\frac{P ( sick )}{P ( healthy )} = \frac{0.01}{0.99} = \frac{1}{99}$ . Step 2 — Bayes factor. $\frac{P ( + ∣ sick )}{P ( + ∣ healthy )} = \frac{0.99}{0.05} = 19.8$ . Step 3 — posterior odds. $19.8 \times \frac{1}{99} = 0.2$ . Step 4 — odds ko probability mein convert karo. $p = \frac{odds}{1 + odds} = \frac{0.2}{1.2} \approx 0.1667$ . Yeh step kyun? Odds aur probability ek hi belief ke liye do dials hain; identity $p = \frac{o}{1 + o}$ bas [!definition] odds relation $o = \frac{p}{1 - p}$ ko $p$ ke liye solve karna hai, toh hum ek familiar percentage report karne ke liye convert karte hain.

Verify: $0.1667$ exactly Ex 1 se match karta hai. ✓ Same physics, arithmetic ki ek line. Yeh woh form hai jo exam time-pressure mein reach karna chahiye.

Recall Main kis cell mein hoon? (self-test — guess karne ke baad reveal karo)

"Rare disease, ek positive test" ::: Cell A/B — discrete, base-rate dominated. "Maine pehle ek baar update kiya aur zyada data aaya" ::: Cell C — posterior naya prior ban jaata hai. "Coin bias, pehle se koi strong opinion nahi" ::: Cell D — continuous, flat Beta(1,1). "Coin bias lekin mujhe sach mein lagta hai yeh near fair hai" ::: Cell E — informative Beta prior. "Mere paas hazaron observations hain" ::: Cell G — data prior ko duba deta hai, MLE ki taraf converge hota hai. "Mujhe sirf do hypotheses fast compare karni hain" ::: Cell I — odds form, evidence cancel ho jaata hai.