Bayesian statistics — prior, likelihood, posterior (intro)

4.9.24 · Maths › Probability Theory & Statistics

YE EXIST KYUN KARTA HAI?

KYA problem solve karta hai: Tumhare paas ek unknown quantity hai (coin ka bias, kisi disease ki probability, ek model parameter $θ$ ). Tum $θ$ ko directly observe nahi kar sakte — tum sirf data $D$ dekhte ho. Bayesian statistics ka jawab hai: jo data maine actually dekha, usse dekh ke ab mujhe $θ$ ke baare mein kya believe karna chahiye?

YE POWERFUL KYUN HAI: Ye unknown $θ$ ko khud ek random variable maanta hai jiske paas ek probability distribution hai. Isse tum ek parameter ke baare mein uncertainty express kar sakte ho, sirf data ke baare mein nahi. Frequentists kehte hain " $θ$ fixed hai par unknown hai"; Bayesians kehte hain " $θ$ fixed hai par iske baare mein meri knowledge ek distribution hai jise main refine karta rehta hoon."

YE KAAM KAISE KARTA HAI — Bayes' theorem ko scratch se derive karna

Step 1 — joint ko do taraf se likho. Joint probability $P (A \cap B)$ order ki parwah nahi karta, isliye: $P (A \cap B) = P (A ∣ B) P (B) = P (B ∣ A) P (A)$ Ye step kyun? Dono expansions same overlap region describe karti hain — ek pehle $B$ se kaati hai, doosri $A$ se.

Step 2 — barabar karo aur divide karo. Dono right-hand sides ko equal set karo aur $P (B)$ se divide karo: $P (A ∣ B) = \frac{P ( B ∣ A ) P ( A )}{P ( B )}$ Ye step kyun? Hum us cheez ko isolate karte hain jo hum chahte hain ( $A$ given $B$ ) un cheezon ke terms mein jo hum pa sakte hain ( $B$ given $A$ , aur priors).

Step 3 — statistics ke liye rename karo. $A = θ$ (hypothesis/parameter) aur $B = D$ (data) rakh do:

Chaar characters

Figure — Bayesian statistics — prior, likelihood, posterior (intro)

Worked Example 1 — Disease testing (discrete)

Ek disease 1% logon ko affect karti hai. Ek test 99% sensitive hai ( $P (+ ∣ sick) = 0.99$ ) aur 95% specific ( $P (- ∣ healthy) = 0.95$ , isliye false-positive rate $0.05$ ). Tumhara test positive aata hai. $P (sick ∣ +)$ kya hai?

Setup. $θ \in {sick, healthy}$ . Prior: $P (sick) = 0.01$ .

Step 1 — likelihoods. $P (+ ∣ sick) = 0.99$ , $P (+ ∣ healthy) = 0.05$ . Kyun? Ye batata hai ki har hypothesis data "test = +" ko kitna achha explain karti hai.

Step 2 — evidence. $P (+) = 0.99 (0.01) + 0.05 (0.99) = 0.0099 + 0.0495 = 0.0594$ Ye step kyun? Positive test aane ki total probability, dono hypotheses (sick aur healthy) par sum ki gayi.

Step 3 — posterior. $P (sick ∣ +) = \frac{0.99 \times 0.01}{0.0594} = \frac{0.0099}{0.0594} \approx 0.1667$

Punchline: Sirf ~17%! "99%" test ke bawajood bhi, tiny prior dominate karta hai. Kyun? Healthy log bahut zyada hain, isliye false positives true positives se zyada hote hain.

Worked Example 2 — Coin bias (continuous, Beta–Binomial)

Unknown bias $θ \in [0, 1]$ . Tum flip karte ho aur 10 tosses mein 7 heads milte hain. Flat prior $P (θ) = 1$ se shuru karo ( $[0, 1]$ par Uniform).

Step 1 — likelihood (Binomial). $P (D ∣ θ) = (7 10) θ^{7} (1 - θ)^{3}$ Kyun? Har head $θ$ contribute karta hai, har tail $(1 - θ)$ ; $(7 10)$ orderings count karta hai ( $θ$ mein constant).

Step 2 — proportional posterior. $P (θ ∣ D) \propto θ^{7} (1 - θ)^{3} \times 1$ Ye step kyun? posterior $\propto$ likelihood $\times$ prior; flat prior $= 1$ nikal jaata hai, aur binomial constant absorb ho jaata hai.

Step 3 — shape pehchano. $θ^{a - 1} (1 - θ)^{b - 1}$ ek Beta distribution hai jisme $a = 8, b = 4$ hai. Isliye $θ ∣ D \sim Beta (8, 4) .$ Kyun? Uniform $Beta (1, 1)$ hai; $7$ heads aur $3$ tails add karne par $Beta (1 + 7, 1 + 3)$ milta hai — Beta Binomial ka conjugate hai (posterior same family mein rehta hai).

Step 4 — point estimate. Posterior mean $= \frac{a}{a + b} = \frac{8}{12} = 0.667$ . Ye kyun achha hai: ye prior mean $0.5$ aur data fraction $0.7$ ke beech baithta hai — prior estimate ko $0.5$ ki taraf thoda shrink karta hai.

Forecast-then-Verify

Common mistakes (Steel-man + fix)

Recall Feynman: ek 12-saal ke bachche ko explain karo (hidden — pehle khud try karo!)

Socho tum sochte ho ki ek cookie jar probably full hai (ye tumhara prior guess hai). Phir tum use hilaate ho aur sirf thodi si khankhanaahat sunti hai. Full jar mein LOUD khankhanaahat hoti, almost-empty jar mein thodi si — ye "is awaaz ki kitni probability hai" hi likelihood hai. Apne pehle guess ko jo suna ussse milao, aur ab tumhe lagta hai jar almost empty hai: ye naya belief hi posterior hai. Bayes' rule bas "jo maine socha" aur "jo maine dekha" ko honestly mix karne ka tarika hai. Aur surprise ye hai: agar tum bahut zyada sure the ki jar full hai (strong prior), to ek thodi khankhanaahat tumhara mann zyada nahi badlegi — strong priors zidd karte hain.

Active-recall flashcards

Prior

P (θ)

kya represent karta hai?

Koi bhi data dekhne se pehle parameter

θ

ke baare mein tumhari belief.

Likelihood

P (D ∣ θ)

kya hai, aur kya ye

θ

ke upar ek distribution hai?

θ

ki har value ke liye observed data kitna probable hai; ye

θ

ka ek function hai lekin

θ

ke upar probability distribution NAHI hai (1 tak integrate hona zaroori nahi).

Ek parameter aur data ke liye Bayes' theorem state karo.

P (θ ∣ D) = \frac{P ( D ∣ θ ) P ( θ )}{P ( D )}

Posterior ka proportional form likho.

P (θ ∣ D) \propto P (D ∣ θ) P (θ)

— posterior ∝ likelihood × prior.

Evidence

P (D)

ka kya role hai?

Ek normalizing constant (jo

θ

se independent hai) jo posterior ko sum/integrate karke 1 banata hai;

P (D) = \int P (D ∣ θ) P (θ) d θ

Disease testing mein 1% prevalence aur 99%/95% test ke saath,

P (sick ∣ +)

sirf ~17% kyun hai?

Low prior ka matlab hai healthy log sick logon se bahut zyada hain, isliye false positives true positives se zyada hote hain.

Beta Binomial ka conjugate kyun hai?

Beta prior aur Binomial likelihood ka product Beta posterior deta hai;

Beta (a, b)

h

heads aur

t

tails ke saath update karne par

Beta (a + h, b + t)

milta hai.

[0, 1]

par Uniform prior kaunsi Beta distribution ke barabar hai?

Beta (1, 1)

Flat prior ke saath 10 tosses mein 7 heads ke baad, posterior aur uska mean kya hai?

Beta (8, 4)

, mean

= 8/12 \approx 0.667

P (θ ∣ D)

aur

P (D ∣ θ)

ko confuse karna kaunsi fallacy hai?

Prosecutor's fallacy (conditioning direction ko ulta samajhna).

Connections

Conditional Probability — wo foundation jisse Bayes derive hota hai.
Law of Total Probability — evidence $P (D)$ kaise compute hoti hai.
Binomial Distribution — coin example mein likelihood.
Beta Distribution — proportions ke liye conjugate prior/posterior.
Maximum Likelihood Estimation — prior ignore karne par (ya flat prior use karke mode lene par) kya milta hai.
Naive Bayes Classifier — is rule ka ek direct ML application.
Frequentist vs Bayesian Inference — philosophical contrast.