Foundations — Maximum a posteriori estimation (MAP)

1.3.17 · D1 · AI-ML › Probability & Statistics › Maximum a posteriori estimation (MAP)

Parent note Maximum a posteriori estimation (MAP) padhne se pehle, tumhe har woh symbol bina jhijhak ke pata hona chahiye jo wo use karta hai. Yeh page har ek ko kuch nahi se build karta hai — pehle plain words, phir ek picture, phir kyun topic ko uski zaroorat hai. Upar se neeche padhna; har block uske upar wale par tika hai.

1. "Parameter" $\theta$ kya hota hai?

Isse picture karo. Ek aisa coin imagine karo jiska heads aane ka chance $0.5$ par fixed nahi hai. Woh hidden chance hi $\theta$ hai. Agar $\theta =0.7$ hai, toh coin heads-biased hai; agar $\theta =0.2$ hai, toh yeh tails-biased hai. $\theta$ yahaan $0$ se $1$ tak ek number line par rehta hai.

Topic ko isko kyun chahiye. MAP (Maximum a Posteriori) ka poora kaam $\theta$ ko estimate karna hai — us line par ek jagah point karke kehna "yeh mera best guess hai." Baaki sab kuch us jagah ko choose karne ki machinery hai.

2. "Hat" $\hat{\theta }$ — $\theta$ ka ek estimate

Isse picture karo. True $\theta$ number line par ek fixed lekin invisible dot hai. Hamaari method ek flag $\hat{\theta }$ gaadi wahin jahan use lagta hai dot hai. Woh miltay nahi bhi ho sakte — hat ek guess hai.

Topic ko isko kyun chahiye. MAP ek particular guess produce karta hai, likha jaata hai ${\hat{\theta }}_{\text{MAP}}$ (padhte hain "theta-hat MAP"): Maximum a Posteriori rule se choose kiya gaya $\theta$ ka estimate. Subscript sirf yeh label karta hai ki kaunsi estimation method ne guess banaya — tum ${\hat{\theta }}_{\text{MLE}}$ bhi ek alag method se miloge.

3. "Data" $\mathbf{x}$ kya hota hai?

Isse picture karo. Tum coin 10 baar flip karte ho aur results likhte ho: $\mathbf{x}=\{H,H,T,H,H,H,T,T,H,H\}$ 10 outcomes ki woh list hi $\mathbf{x}$ hai, aur yahaan $n=10$.

Topic ko isko kyun chahiye. $\theta$ invisible hai; $\mathbf{x}$ woh hai jo hum dekh sakte hain. MAP (Maximum a Posteriori) woh bridge hai jo visible data se hidden knob tak jaata hai.

4. Probability $P(\cdot )$ aur pipe $\mid$

Isse picture karo. $P(\text{rain})$ $0.3$ ho sakta hai. Lekin $P(\text{rain}\mid \text{dark clouds})$ $0.8$ tak jump karta hai — clouds ke baare mein jaanna probability badal deta hai. Pipe ek filter hai: woh har woh world throw karta hai jahan $B$ false hai, phir jo bachta hai usme $A$ measure karta hai.

Topic ko isko kyun chahiye. MAP do conditional probabilities par bana hai jo ulti disha mein point karti hain:

• $P(\mathbf{x}\mid \theta )$ — "agar knob $\theta$ hoti, toh yeh data kitna likely hai?"
• $P(\theta \mid \mathbf{x})$ — "yeh data dekhte hue, knob $\theta$ hone ki kitni probability hai?"

Poora subject pehle ko doosre mein flip karne ke baare mein hai.

5. Independence aur i.i.d. data

Probabilities multiply karne se pehle, hume jaanna hai ki yeh kab legal hai.

Isse picture karo. Das alag coin flips: 3rd flip ko koi fark nahi ki 2nd ne kya kiya, aur har flip same biased coin use karta hai. Yahi exactly i.i.d. hai.

Topic ko isko kyun chahiye. Independence woh permission slip hai jo joint probability ko ek product mein factor karne deta hai: $P(x_1\text{ and }{x}_2)=P(x_1)P(x_2)$. Iske bina, Section 11 ka neat $\prod$ notation galat hoga. Parent ke har worked example mein silently i.i.d. data assume kiya gaya hai.

6. Teen named probabilities

Upar ke har conditional ka ek naam hai. Inhe faces ki tarah seekho.

Isse picture karo. Ek wide, vague "prior" hump se $\theta$ ke upar shuru karo. Data use reshape karke ek sharper "posterior" hump bana deta hai. Us final hump ka peak MAP estimate ${\hat{\theta }}_{\text{MAP}}$ hai.

"Posterior" kyun target hai. Yeh teeno mein se ek hi hai jo actual question ka jawab deta hai: ab jo kuch mujhe pata hai, $\theta$ kya hai? Likelihood prior knowledge ignore karta hai; prior data ignore karta hai; posterior dono ko fuse karta hai.

7. Bayes' theorem — the flipping machine

Plain words mein. Posterior equals likelihood times prior, evidence $P(\mathbf{x})$ se divide kiya.

Kyun chahiye. Hum $P(\mathbf{x}\mid \theta )$ compute kar sakte hain (models batate hain ki ek diye hue knob ke liye data kitna likely hai). Hum $P(\theta \mid \mathbf{x})$ chahte hain. Bayes' theorem exactly woh machinery hai jo jo hum compute kar sakte hain use jo hum chahte hain mein convert karti hai. Dekho Bayes Theorem.

8. $\arg \max$ operator

Isse picture karo. Posterior-vs-$\theta$ ke ek curve par, $\max$ summit ki height hai; $\arg \max$ woh $\theta$-coordinate hai jo seedha summit ke neeche hai. MAP location chahta hai, isliye woh $\arg \max$ use karta hai.

Topic ko isko kyun chahiye. "Most probable parameter" ka literally matlab hai "woh $\theta$ jahan posterior sabse tall hai" — yahi exactly ${\hat{\theta }}_{\text{MAP}}=\arg {\max }_{\theta }P(\theta \mid \mathbf{x})$ hai.

9. Logarithm $\log$ — kyun appear karta hai

Topic ko isko kyun chahiye. Likelihoods bahut si chhoti probabilities ka product hoti hain (ek har data point ke liye). Zero ke paas hundreds of numbers multiply karna computer ko underflow karke $0$ kar deta hai. $\log$ lena product ko ek friendly sum mein convert karta hai, aur kyunki $\log$ monotonic hai, peak ki location nahi hilti. Toh: ${\hat{\theta }}_{\text{MAP}}=\arg {\max }_{\theta }[P(\mathbf{x}\mid \theta )P(\theta )]=\arg {\max }_{\theta }[\log P(\mathbf{x}\mid \theta )+\log P(\theta )]$ Same answer, numerically safe, easier calculus.

10. Derivative $\frac{d}{d\theta }$ — peak dhundhna

Topic ko isko kyun chahiye. Ek smooth hill ke bilkul top par, ground momentarily flat hota hai — slope $=0$. Toh log-posterior ka peak dhundhne ke liye hum iska derivative zero set karte hain aur solve karte hain. Isliye har worked example "$\frac{d}{d\theta }=0$" par khatam hota hai. Flat-top rule hi woh hai jisse hum $\arg \max$ algebraically locate karte hain.

11. Product $\prod$ aur sum $\sum$ notation

Topic ko isko kyun chahiye. Kyunki data i.i.d. hai (Section 5), joint likelihood har point ki likelihood ke product mein factor ho jaati hai — isliye $\prod$. $\log$ lene ke baad (Section 9), woh product ek $\sum$ ban jaata hai. Yahi exact move parent ke Gaussian example mein hai.

Prerequisite map

Har arrow ka matlab hai "tail samajhne ke baad hi head make sense karta hai." Note karo ki likelihood aur prior dono Bayes' theorem ko feed karte hain, jo posterior produce karta hai, jise $\arg \max$ + log + derivative finally MAP estimate ${\hat{\theta }}_{\text{MAP}}$ mein squeeze karte hain.

Yeh aage kahan le jaate hain

Jab yeh symbols solid ho jaayein, tab tum padh sakte ho ki MAP (Maximum a Posteriori) bahar kaise connect karta hai:

• Maximum Likelihood Estimation (MLE) — flat prior wala MAP. Flat (ya uniform) prior woh hai jo har allowed $\theta$ ko same $P(\theta )$ assign karta hai; constant hone ki wajah se, yeh $\arg \max$ mein kuch add nahi karta, toh MAP pure likelihood maximization mein wapas collapse ho jaata hai.
• Conjugate Priors aur Beta Distribution — priors jo algebra clean rakhte hain.
• Regularization in ML, Ridge Regression, Lasso Regression — MAP disguise mein.
• Overfitting — woh bimari jise prior theek karta hai.

Equipment checklist

Right side cover karo aur har ek ka jawab do. Agar ruk jaao, woh section dobara padhna.