Bayesian statistics — prior, likelihood, posterior (intro): Foundations

4.9.24 · D1 · Maths › Probability Theory & Statistics › Bayesian statistics — prior, likelihood, posterior (intro)

Beliefs update karne se pehle, tumhe woh sentence padhni aani chahiye jo update karti hai. Yeh page har ek notation ka piece build karta hai jo parent note silently assume karta hai, ek aisi order mein jahan har symbol apne pehle wale symbol se earn hota hai. Kisi bhi aisi symbol ke aage mat badho jise tum picture nahi kar sakte.

1. Probability kya hoti hai? Symbol $P (\cdot)$

Picture. Ek bada rectangle banao aur uski poori area ko $1$ kaho — yeh rectangle sab kuch jo ho sakta hai hai (the sample space). Ek event $A$ uske andar ek blob hai. $P (A)$ simply rectangle ka woh fraction hai jo blob cover karta hai.

Yeh topic isko kyun use karta hai. Bayesian statistics ek machine hai jo probabilities andar leti hai aur probability bahar deti hai. Agar "ek number jo measure karta hai ki ek blob kitna area cover karta hai" bilkul crystal clear nahi hai, toh baad ke koi bhi symbols kuch matlab nahi rakhte.

2. Do events ek saath: $\cap$ aur joint region

Picture. Rectangle ke andar do overlapping blobs banao, $A$ aur $B$ . Lens-shaped overlap hi $A \cap B$ hai. Uski area $P (A \cap B)$ hai — woh chance ki tum dono blobs mein land karo.

Yeh topic isko kyun use karta hai. Parent ka Step 1 "joint do tarike se" likhta hai: $P (A \cap B) = P (A ∣ B) P (B) = P (B ∣ A) P (A) .$ Woh poori derivation ek statement hai is ek overlapping lens ke baare mein jo do directions se measure ki jaati hai. Overlap ki picture ke bina, woh equation sirf symbols hai.

3. Conditioning: bar $∣$ aur $P (A ∣ B)$

Kyun yahi formula aur koi nahi? Jab hum jaante hain ki $B$ hua, toh duniya chhoti ho gayi: sirf $B$ -blob ab possible hai. Toh hum rectangle ka baaki hissa phek dete hain aur poochte hain is nayi chhoti duniya ka (area $P (B)$ ) kitna fraction bhi $A$ mein hai (area $P (A \cap B)$ )? Divide karna overlap ko rescale karta hai taaki $B$ -blob ab poore "certain" amount $1$ ki tarah count ho.

Figure dekho. Left mein, $A \cap B$ poori rectangle ka ek chhota sa slice hai. Right mein hum zoom kar lete hain taaki $B$ frame fill kare — wahi slice ab bahut bada lagta hai, kyunki humne chhoti duniya $P (B)$ se divide kiya.

4. Unknowns ko naam dena: $θ$ aur $D$

Picture. $θ$ ko ek aise dial ki tarah socho jo ek locked box ke andar chhupa hua hai, aur $D$ woh readings hain jo ek window se bahar leak hoti hain. Hum dial ko kabhi touch nahi karte; hum sirf readings dekhte hain aur dial ki position ki taraf backwards reason karte hain.

Yeh topic isko kyun use karta hai. Poora subject arrow $D \Rightarrow θ$ hai: visible readings se hidden dial tak. Bayes' theorem literally woh machine hai jo arrow ko reverse karti hai, kyunki nature isse aage chalati hai ( $θ$ causes $D$ ) aur hum ise ulta chahte hain.

5. Chaar characters symbols ki tarah

Ab parent ke headline formula mein har ek symbol ek aisi phrase hai jo tum zor se bol sakte ho.

Symbol	Words mein kaho	Picture
$P (θ)$	koi reading aane se pehle dial mein belief	dial kitna ho sakta hai, pehle se guess kiya hua
$P (D ∣ θ)$	agar dial $θ$ par hota, toh yeh reading kitni likely hai	forward arrow: dial → reading
$P (D)$	sab dial settings par yeh reading kitni likely hai	total window brightness, averaged
$P (θ ∣ D)$	reading dekhne ke baad dial mein belief	backward arrow: reading → dial

$P (D)$ "sirf ek normalizer" kyun hai. Isme koi $θ$ nahi hai, isliye jab $θ$ vary karta hai yeh ek fixed constant hai. Iska sirf ek kaam hai: posterior ki total area ko $1$ banao (ek real probability). Isliye parent's proportional form ise drop karta hai: $P (θ ∣ D) \propto P (D ∣ θ) P (θ)$ .

6. Summation, integration, aur $\propto$

Do symbols tab aate hain jab hum evidence $P (D)$ build karte hain.

Picture. Discrete $θ$ : kuch bars jo tum add karte ho. Continuous $θ$ : ek smooth curve jiska area neeche tum accumulate karte ho. Integral "curve ke neeche ka area" hai.

Yeh topic dono kyun use karta hai. Law of Total Probability precisely $P (D) = \sum_{θ} P (D ∣ θ) P (θ)$ hai (ya integral version) — yahi woh hai ki denominator kaise assemble hota hai. Aur $\propto$ parent ka "80/20" hai: un-normalized shape compute karo, phir constant ko bilkul end mein fix karo.

7. Do data-generators jinpar parent rely karta hai

Abhi kyun, pehle kyun nahi. Yeh Worked Example 2 mein specific likelihood aur prior shapes hain. Tumhe pehle $θ$ , $\propto$ , aur "curve ke neeche area" chahiye tha taaki " $θ^{7} (1 - θ)^{3}$ ek Beta hai" kuch matlab rakh sake. Magic word conjugate (Beta prior + Binomial data → Beta posterior) bas yeh hai: shapes multiply hoti hain aur same family mein rehti hain.

8. Prerequisite map

Equipment checklist

Answers cover karo aur khud test karo. Agar koi bhi line confuse kare, toh uska section upar se dobara padho.

P (A)

kya measure karta hai, aur uska range kya hai?

Woh fraction of the sample-space rectangle jo event

A

cover karta hai;

[0, 1]

mein ek number.

A \cap B

ka matlab kya hai aur woh kaisa dikhta hai?

"Dono

A

aur

B

"; do blobs ka overlapping lens.

Conditional formula

P (B)

se kyun divide karta hai?

Kyunki

B

jaanna duniya ko

B

-blob tak chhota kar deta hai, isliye hum overlap ko rescale karte hain taaki

B

ko nayi poori "certain" area treat kar sakein.

Kya

P (A ∣ B)

aur

P (B ∣ A)

same hain?

Nahi — same overlap, alag denominators; ek ko doosre mein convert karna exactly Bayes' theorem hai.

θ

versus

D

kya hai?

θ

woh hidden unknown hai jo hum chahte hain (the dial);

D

woh observed data hai (the readings).

Bayes' theorem ke chaar characters ke naam batao.

Prior

P (θ)

, likelihood

P (D ∣ θ)

, posterior

P (θ ∣ D)

, evidence

P (D)

Proportional form mein hum

P (D)

kyun drop kar sakte hain?

Isme koi

θ

nahi hai, isliye yeh ek constant hai; yeh sirf total area ko

1

tak rescale karta hai aur end mein restore kiya ja sakta hai.

\sum_{θ}

aur

\int d θ

mein kya fark hai?

Dono

θ

ke saare values par add karte hain;

\sum

discrete list ke liye,

\int

continuous range ke liye (area under a curve).

f \propto g

tumhe kya batata hai?

f

ki shape

g

jaisi hi hai ek unknown constant factor tak.

n

tosses mein

k

heads ke liye Binomial likelihood kya hai?

(k n) θ^{k} (1 - θ)^{n - k}