Maximum a posteriori estimation (MAP): Worked examples

1.3.17 · D3 · AI-ML › Probability & Statistics › Maximum a posteriori estimation (MAP)

Yeh page ek drill hai. Iska parent note, MAP estimation, tumhe machinery dikha chuka hai: $\hat{θ}_{MAP} = ar g max_{θ} [lo g P (x ∣ θ) + lo g P (θ)]$ . Yahan hum har tarah ki situation us machinery par throw karte hain taaki koi bhi exam question tumhe surprise na kar sake.

Pehle, ek symbol jo hum har line par use karenge:

Ab words ki ek chhoti si reminder, plain language mein, har ek ko neeche ki ek picture se anchor karke:

MAP bas itna hai: "likelihood times prior ka sabse ooncha point dhundo." Neeche sab kuch wahi ek sentence hai, stress-tested.

Scenario matrix

Har MAP problem neeche ke aath cells A–H mein se kisi ek mein aata hai. Table ko aise padho: "cell → kya cheez usse tricky banati hai → kaun sa example usse drill karta hai." Baad mein har worked example apne cell letter ke saath tagged hai, taaki tum kisi bhi scenario ko uske label se dhundh sako. A–H ko ek checklist ki tarah socho: agar tum sabhon ko kar sakte ho, toh koi bhi MAP question naya nahi hai.

Cell	Cell class	Kya cheez tricky banati hai	Example
A	Prior data se disagreement karta hai	prior estimate ko ek taraf kheenchta hai, data doosri taraf	Ex 1
B	Data-scarce vs data-rich limit	chhota $n$ (prior dominate karta hai) → bada $n$ (data dominate karta hai)	Ex 2
C	Degenerate: zero successes ( $k = 0$ )	MLE $0$ deta hai; kya MAP bhi?	Ex 3
D	Degenerate: uniform prior	MAP exactly MLE ke barabar collapse hona chahiye	Ex 4
E	Mode $\neq =$ mean (asymmetric posterior)	kaun sa number report karna hai	Ex 5
F	Regularization face (Gaussian prior = L2)	penalty ko prior bante dekho	Ex 6
G	Real-world word problem	English ko likelihood + prior mein translate karo	Ex 7
H	Exam twist: prior with $α < 1$ or $β < 1$ jo boundary solution force karta hai	derivative andar kabhi zero nahi hoti; edges check karo	Ex 8

Prerequisites in reach: Bayes Theorem, Beta Distribution, Posterior Distribution, Conjugate Priors, Maximum Likelihood Estimation (MLE), Regularization in ML.

Figure — Maximum a posteriori estimation (MAP)

Figure 1 (pehle mujhe padho). Blue curve likelihood $P (x ∣ θ)$ hai, pink curve prior $P (θ)$ hai, aur yellow curve unka point-by-point product, yaani posterior hai. Dashed yellow line uske peak ko mark karti hai — yahi MAP estimate hai. Yeh akela picture Examples 1–5 aur 8 ka template hai: woh sab is $[0, 1]$ axis par rehte hain, bas alag blue aur pink shapes ke saath. Ise apne paas rakho.

Coin-flip cells (Beta prior + Binomial data)

Jab bhi data "trials mein se successes" ho, likelihood Binomial hoti hai aur natural prior ek Beta hota hai. Beta $(α, β)$ ki shape yaad karo: $P (θ) \propto θ^{α - 1} (1 - θ)^{β - 1}$ for $θ \in [0, 1]$ . Kyunki Beta, Binomial ka conjugate prior hai, posterior bhi Beta hota hai:

Yeh mode formula kyun, aur edge checks kyun? $lo g [θ^{a - 1} (1 - θ)^{b - 1}]$ lo aur differentiate karo: slope hai $\frac{a - 1}{θ} - \frac{b - 1}{1 - θ}$ . Ise $0$ set karne par woh fraction milta hai. Lekin ek stationary point tabhi andar $(0, 1)$ mein exist karta hai jab dono $a - 1$ aur $b - 1$ ka sign aisa ho ki dono terms trade off karein. Agar, maano, $a - 1 \leq 0$ hai, toh pehla term slope ko kabhi positive nahi kheenchta, isliye function sirf girta hai — uska sabse ooncha point boundary $θ = 0$ hai. Isliye blindly plug in nahi kar sakte: " $- 1$ "s tumhe edge par dhakka de sakti hain. (Wahi " $- 1$ " mismatch mode ko mean se alag bhi banata hai — ek distinction jo Ex 5 achhi tarah hammer karta hai.)

Worked example Example 1 — Cell A: prior data se ladhta hai

Statement: 10 flips mein 8 heads, toh $x$ mein $n = 10$ entries hain jisme $k = 8$ heads hain. Prior Beta $(2, 8)$ hai (tumhara strong belief tha ki coin tails favour karta hai). $\hat{θ}_{MAP}$ nikalo aur MLE se compare karo.

Forecast: MLE $0.8$ hai. Prior mean $2/10 = 0.2$ hai. Guess: MAP kahin beech mein utarega, $0.8$ ke kareeb kyunki 10 flips real evidence hai. Apna guess likh lo.

$k, n, α, β$ identify karo. $k = 8, n = 10, α = 2, β = 8$ . Yeh step kyun? Formula ko sirf yeh char numbers chahiye.
Edges check karo, phir plug in karo. Posterior shape numbers $a = k + α = 10$ , $b = n - k + β = 10$ ; dono $> 1$ , isliye interior formula valid hai: $\hat{θ}_{MAP} = \frac{10 - 1}{10 + 10 - 2} = \frac{9}{18} = 0.5$ . Yeh step kyun? Hum boxed rule maan rahe hain — fraction trust karne se pehle $a > 1, b > 1$ confirm karo.
Compare karo. $\hat{θ}_{MLE} = 8/10 = 0.8$ ; prior mean $0.2$ ; MAP $= 0.5$ . Yeh step kyun? Tug-of-war dekhne ke liye: MAP data aur prior ke beech baitha hai.

Verify karo: $0.5$ strictly $0.2$ aur $0.8$ ke beech hai — ek proper compromise. Sanity check: posterior Beta $(10, 10)$ hai, symmetric, toh uska mode $= 0.5$ hai. ✓ Yeh Figure 1 ke template se match karta hai: right-leaning blue likelihood aur left-leaning pink prior blend hokar centred yellow peak banate hain.

Worked example Example 2 — Cell B: dekho prior fade hota hai jaise data badhta hai

Statement: Coin ka true head-rate $70%$ heads produce karta rehta hai. Prior Beta $(3, 2)$ hai. $n = 10$ (toh $k = 7$ ) aur $n = 1000$ (toh $k = 700$ ) ke liye $\hat{θ}_{MAP}$ compute karo.

Forecast: Kam data ke saath prior mean $0.6$ estimate ko neeche kheenchta hai; bahut zyada data ke saath estimate $0.7$ par snap hona chahiye. Dono numbers guess karo.

Chhota $n$ . Shape numbers $a = 7 + 3 = 10$ , $b = 3 + 2 = 5$ , dono $> 1$ , toh $\hat{θ}_{MAP} = \frac{10 - 1}{10 + 5 - 2} = \frac{9}{13} \approx 0.6923$ . Yeh step kyun? Hamare formula par ek pehla checkpoint; yeh parent note ke coin example ki value reproduce karta hai.
Bada $n$ . $a = 700 + 3 = 703$ , $b = 300 + 2 = 302$ , dono $> 1$ , toh $\hat{θ}_{MAP} = \frac{703 - 1}{703 + 302 - 2} = \frac{702}{1003} \approx 0.6999$ . Yeh step kyun? Wahi formula; sirf $k, n$ change hote hain, toh hum dekhte hain ki fixed prior constants ( $α, β$ ) negligible ho jaate hain.
Trend padho. $0.6923 \to 0.6999$ jab $n$ 100 se multiply hota hai — estimate MLE $0.7$ ki taraf march karta hai. Yeh step kyun? Yahi general law hai: jaise $n \to \infty$ , MAP $\to$ MLE.

Verify karo: $702/1003 = 0.69990 \dots$ , aur $∣0.6999 - 0.7∣ = 0.0001$ , small- $n$ gap $∣0.6923 - 0.7∣ = 0.0077$ se kaafi chhota hai. Prior ka influence ~77× shrink hua. ✓

Figure 2 (yeh Example 2 hai). Har curve Example 2 ki ek sample size ke liye posterior hai. Jaise $n$ badhta hai (blue → pink → yellow), peak $\approx 0.69$ se $0.70$ (dashed white MLE line) ki taraf slide hoti hai aur narrow hoti hai — zyada data accuracy aur confidence dono deta hai. Dotted vertical lines woh teen MAP peaks hain jo tumne upar compute kiye.

Worked example Example 3 — Cell C: zero successes (ek degenerate input)

Statement: 5 flips mein 0 heads ( $x$ mein paanch tails hain). Prior Beta $(2, 2)$ hai. $\hat{θ}_{MAP}$ nikalo aur note karo ki MLE kyun misbehave karta hai.

Forecast: MLE chillaata hai $θ = 0/5 = 0$ ("yeh coin kabhi heads nahi aata") — 5 flips se ek overconfident claim. Guess karo ki MAP exactly $0$ hai ya kuch positive.

Pehle MLE. $\hat{θ}_{MLE} = 0/5 = 0$ . Yeh step kyun? Pathology expose karne ke liye: heads ki zero probability hamesha ke liye, mutthi bhar flips se. Yeh sparse data par overfitting hai.
Edges check karo, phir MAP. Shape numbers $a = 0 + 2 = 2$ , $b = 5 + 2 = 7$ , dono $> 1$ , toh interior formula apply hota hai: $\hat{θ}_{MAP} = \frac{2 - 1}{2 + 7 - 2} = \frac{1}{7} \approx 0.1429$ . Yeh step kyun? Beta $(2, 2)$ ek "pseudo-head" aur ek "pseudo-tail" add karta hai, toh $a > 1$ aur estimate boundary se door rehta hai — $k = 0$ ke bawajood yahan koi edge case nahi.
Interpret karo. Prior "impossible heads" mein believe karne se inkar karta hai, estimate ko ek modest $0.143$ tak nudge karta hai. Yeh step kyun? Yahi exactly reason hai ki hum priors kyun pasand karte hain: woh absurd boundary answers se regularize karte hain.

Verify karo: $1/7 \approx 0.1429 > 0$ , strictly $(0, 1)$ ke andar, MLE ke boundary $0$ se alag. Shape numbers $a = 2 > 1$ , $b = 7 > 1$ , toh interior formula sahi tool tha. ✓

Worked example Example 4 — Cell D: uniform prior ko MLE wapas dena hi hoga

Statement: 10 flips mein 7 heads. Prior Beta $(1, 1)$ ( $[0, 1]$ par uniform prior). Dikhao ki $\hat{θ}_{MAP} = \hat{θ}_{MLE}$ .

Forecast: Beta $(1, 1)$ flat hai — pehle se har $θ$ equally believable. Toh MAP exactly MLE reproduce karna chahiye: guess $0.7$ .

Flat prior recognize karo. $θ^{1 - 1} (1 - θ)^{1 - 1} = θ^{0} (1 - θ)^{0} = 1$ sab $θ$ ke liye. Yeh step kyun? Ek constant prior log-posterior mein ek constant add karta hai, jo argmax ko move nahi kar sakta.
Edges check karo, phir formula apply karo. Shape numbers $a = 7 + 1 = 8$ , $b = 3 + 1 = 4$ , dono $> 1$ , toh $\hat{θ}_{MAP} = \frac{8 - 1}{8 + 4 - 2} = \frac{7}{10} = 0.7$ . Yeh step kyun? " $+ α - 1$ " aur " $+ β - 1$ " terms $α = β = 1$ ke liye vanish ho jaate hain, exactly $k / n$ bachta hai.
Identity confirm karo. $\hat{θ}_{MLE} = 7/10 = 0.7 = \hat{θ}_{MAP}$ . Yeh step kyun? Yeh parent claim prove karta hai: MLE, MAP hai uniform prior ke saath.

Verify karo: $7/10 = 0.7$ construction se MLE ke barabar hai. ✓ (Dekho Maximum Likelihood Estimation (MLE).)

Worked example Example 5 — Cell E: mode ≠ mean, tum kya report karte ho?

Statement: Posterior Beta $(2, 5)$ hai. MAP (mode) aur posterior mean dono report karo, aur explain karo ki difference kab problem banata hai.

Forecast: Skewed distribution ke liye yeh agree nahi karte. Mass chhote $θ$ ki taraf lean karta hai, toh mode $<$ mean ya mode $>$ mean? Compute karne se pehle guess karo.

Mode (MAP). Shape numbers $a = 2, b = 5$ , dono $> 1$ , toh mode $= \frac{a - 1}{a + b - 2} = \frac{1}{5} = 0.2$ . Yeh step kyun? MAP peak hai; interior formula apply hota hai kyunki $a > 1, b > 1$ .
Mean (MMSE point). Mean $= \frac{a}{a + b} = \frac{2}{7} \approx 0.2857$ . Yeh step kyun? Posterior mean woh estimate hai jo mean-squared error minimize karta hai — yeh "peak kahan hai?" se ek alag sawaal hai.
Diagnose karo. $0.2 \neq = 0.2857$ : lamba right tail mean ko mode se upar kheenchta hai. Yeh step kyun? MAP report karna jab audience mean expect kare silently statistic change kar deta hai.

Verify karo: Mode $0.2$ , mean $0.2857$ ; gap $0.0857 \neq = 0$ asymmetry confirm karta hai. Parent note ke stated values ( $1/5$ aur $2/7$ ) se cross-check karo. ✓

Gaussian / regularization cells

Gaussian noise wale continuous parameters ke liye, likelihood peak sample mean $\overset{x}{ˉ}$ hota hai, aur ek Gaussian prior use prior mean ki taraf shrink karta hai. Yeh shrinkage hai hi L2 regularization — dekho Regularization in ML, Ridge Regression.

Worked example Example 6 — Cell F: L2 penalty literally ek prior hai

Statement: $N (μ, 1)$ se ek observation $x = 2$ (toh $x = {2}$ , $n = 1$ ). Gaussian prior $μ \sim N (0, σ_{0}^{2})$ with $σ_{0}^{2} = 4$ . $\overset{μ}{^}_{MAP}$ nikalo, aur ridge penalty $λ$ identify karo jisko yeh correspond karta hai.

Forecast: MLE $\overset{μ}{^} = x = 2$ kahega. $0$ par prior use ise shrink karta hai. $(0, 2)$ mein koi number guess karo.

Log-posterior. $lo g P (μ ∣ x) = - \frac{1}{2} (x - μ)^{2} - \frac{μ ^{2}}{2 σ _{0}^{2}} + const$ . Yeh step kyun? Log-likelihood (data fit) aur log-prior (large $μ$ par penalty) ka sum.
Differentiate karo, $0$ set karo. $(x - μ) - \frac{μ}{σ _{0}^{2}} = 0$ . Yeh step kyun? Peak par slope zero hoti hai; yeh equation optimal $μ$ pin karta hai.
$μ$ ke liye rearrange karo. $μ$ terms group karo: $x = μ + \frac{μ}{σ _{0}^{2}} = μ (1 + \frac{1}{σ _{0}^{2}}) = μ \cdot \frac{σ _{0}^{2} + 1}{σ _{0}^{2}}$ , toh $\overset{μ}{^} = \frac{σ _{0}^{2} x}{σ _{0}^{2} + 1} = \frac{4 \cdot 2}{4 + 1} = \frac{8}{5} = 1.6$ . Yeh step kyun? Har $μ$ ko ek side collect karke factor out karo, toh equation " $x = μ \times (constant)$ " ban jaata hai aur divide karne par akela $μ$ milta hai — pure algebra, koi nayi idea nahi.
Penalty read off karo. Prior term $\frac{μ ^{2}}{2 σ _{0}^{2}} = λ μ^{2}$ with $λ = \frac{1}{2 σ _{0}^{2}} = \frac{1}{8} = 0.125$ . Yeh step kyun? Yeh MAP-with-Gaussian-prior ko ridge regression ke roop mein expose karta hai jahan $λ = 1/ (2 σ_{0}^{2})$ hai.

Verify karo: $\overset{μ}{^} = 8/5 = 1.6 \in (0, 2)$ — MLE $2$ se prior mean $0$ ki taraf shrunk, jaisa promise kiya tha. Aur $λ = 1/8 = 0.125$ . ✓

Worked example Example 7 — Cell G: real-world word problem

Statement: Ek factory sensor kisi object ki length $x = 5.2$ cm read karta hai (measurement noise $σ = 1$ cm, toh $N (μ, 1)$ ; yahan $x = {5.2}$ ). Blueprints kehte hain lengths $μ_{0} = 5.0$ cm ke aaspaas cluster karti hain prior std $σ_{0} = 0.5$ cm ( $σ_{0}^{2} = 0.25$ ) ke saath. MAP length estimate kya hai?

Forecast: Reading $5.2$ vs blueprint $5.0$ : prior tight hai ( $σ_{0} = 0.5$ ), measurement noise looser hai ( $σ = 1$ ). Toh MAP blueprint ki taraf lean karna chahiye. Guess karo $5.0$ ke kareeb hoga, $5.2$ se zyada.

English ko likelihood mein translate karo. "Sensor $5.2$ read karta hai noise $σ = 1$ ke saath" matlab reading $N (μ, 1)$ se draw ki gayi hai, toh $lo g P (x ∣ μ) = - \frac{1}{2} (x - μ)^{2} + const$ . Yeh step kyun? "Measurement noise" word ek Gaussian likelihood ka code hai jo true length $μ$ par centred hoti hai; yahi hai ek sentence ko math mein turn karna.
English ko prior mein translate karo. "Blueprints $5.0$ ke aaspaas cluster karti hain std $0.5$ ke saath" matlab $μ \sim N (μ_{0}, σ_{0}^{2})$ with $μ_{0} = 5.0, σ_{0}^{2} = 0.25$ , giving $lo g P (μ) = - \frac{( μ - μ _{0} ) ^{2}}{2 σ _{0}^{2}} + const$ . Yeh step kyun? Part ke design ke baare mein prior knowledge prior distribution ban jaati hai; tight std confident prior matlab.
Log-posterior likho. $lo g P (μ ∣ x) = - \frac{1}{2} (x - μ)^{2} - \frac{( μ - μ _{0} ) ^{2}}{2 σ _{0}^{2}} + const$ . Yeh step kyun? MAP = log-likelihood + log-prior maximize karo; hum sirf steps 1–2 ke do pieces add karte hain.
Differentiate karo aur $0$ set karo. $\frac{d}{d μ} = (x - μ) - \frac{μ - μ _{0}}{σ _{0}^{2}} = 0$ . Yeh step kyun? Posterior ke peak par zero slope hoti hai; yeh equation usse locate karta hai.
$μ$ ke liye rearrange karo. $σ_{0}^{2}$ se multiply karo: $σ_{0}^{2} (x - μ) - (μ - μ_{0}) = 0$ , yaani $σ_{0}^{2} x + μ_{0} = μ (σ_{0}^{2} + 1)$ , toh $\overset{μ}{^} = \frac{σ _{0}^{2} x + μ _{0}}{σ _{0}^{2} + 1}$ . Yeh step kyun? Fraction clear karke aur $μ$ terms collect karke equation " $stuff = μ \times (constant)$ " ban jaati hai; divide karne par $μ$ isolate hota hai. Result data aur prior mean ka ek precision-weighted average hai.
Numbers plug karo. $\overset{μ}{^} = \frac{0.25 \cdot 5.2 + 5.0}{0.25 + 1} = \frac{1.3 + 5.0}{1.25} = \frac{6.3}{1.25} = 5.04$ cm. Yeh step kyun? Jo formula abhi derive kiya usmein direct substitution.

Verify karo: $6.3/1.25 = 5.04$ cm, aur yeh $[5.0, 5.2]$ mein hai, tighter-prior end ke kareeb. Precision weights: prior $1/0.25 = 4$ , data $1/1 = 1$ , ratio $4 : 1$ jo strong pull to $5.0$ se match karta hai. Units: har term cm mein hai, toh answer cm mein hai. ✓

Worked example Example 8 — Cell H:

$α < 1$ wala prior ek boundary solution force karta hai Statement: 3 flips mein 3 heads. Prior Beta $(1, 1)$ (uniform). Yeh bhi consider karo ki Prior Beta $(0.5, 1)$ kya karega. Har ek mein $\hat{θ}_{MAP}$ nikalo. (Twist: derivative $(0, 1)$ ke andar kabhi zero equal nahi ho sakti — max phir kahan rehta hai?)

Forecast: Har flip heads tha, toh likelihood $θ^{3}$ $θ = 1$ ki taraf badhti rehti hai. Guess: peak boundary par hai, interior stationary point par nahi.

Uniform prior objective. Posterior $\propto θ^{3} (1 - θ)^{0} = θ^{3}$ for $θ \in [0, 1]$ ; shape numbers $a = 3 + 1 = 4$ , $b = 0 + 1 = 1$ . Yeh step kyun? $k = n$ aur flat prior ke saath hume $θ$ ki ek pure increasing power milti hai, aur $b = 1$ (not $> 1$ ) ek warning flag hai.
Edge rule apply karo. Kyunki $b = 1 \leq 1$ , boxed rule kehta hai peak right edge par baithti hai: $\hat{θ}_{MAP} = 1$ . (Check karo: $\frac{d}{d θ} θ^{3} = 3 θ^{2} = 0$ sirf $θ = 0$ par, ek minimum, koi interior max nahi confirm karta.) Yeh step kyun? Yahi Cell H ka poora point hai — "derivative zero set karo" fail hota hai, toh hum instead boundary check maan lete hain.
$α < 1$ variant. Prior Beta $(0.5, 1)$ posterior deta hai $\propto θ^{3 - 0.5} (1 - θ)^{0} = θ^{2.5}$ , shape numbers $a = 3.5 > 1$ , $b = 1 \leq 1$ — ab bhi right-edge peak, $\hat{θ}_{MAP} = 1$ . Agar instead humara $k = 0$ hota $α = 0.5$ prior ke saath, toh $a = 0.5 \leq 1$ $\hat{θ}_{MAP} = 0$ force karta (left edge). Yeh step kyun? Dono edge directions dikhane ke liye: $a \leq 1 \Rightarrow 0$ , $b \leq 1 \Rightarrow 1$ .

Verify karo: $θ^{3}$ on $[0, 1]$ sabse bada $θ = 1$ par hai (value $1$ ), kisi bhi interior point jaise $0. 9^{3} = 0.729$ se bada. Toh $\hat{θ}_{MAP} = 1$ . Ek stronger interior prior jaise Beta $(2, 2)$ instead $\frac{3 + 2 - 1}{3 + 2 + 2 - 2} = \frac{4}{5} = 0.8$ deta, overconfident boundary answer ko soften karta. ✓

Recall Quick self-test (answers chhupa lo)

MAP kya report karta hai, mode ya mean? ::: Mode (peak) of the posterior. Beta $(1, 1)$ prior MAP ko kya bana deta hai? ::: Exactly MLE (uniform prior). Beta mode formula use karne se pehle kya check karna chahiye? ::: Ki dono posterior shape numbers 1 se zyada hain ( $a > 1$ aur $b > 1$ ); warna peak ek edge par hai ( $0$ ya $1$ ). $n$ flips mein zero heads: MLE kyun dangerous hai par MAP safe? ::: MLE $0$ deta hai (impossible-heads-forever); Beta prior with $α > 1$ $a > 1$ rakhta hai toh MAP $(0, 1)$ ke andar rehta hai. Strength $λ$ wala L2 regularization kaun se prior ke saath MAP hai? ::: Gaussian $N (0, σ^{2})$ with $λ = 1/ (2 σ^{2})$ . "Derivative zero set karo" step kab fail hota hai? ::: Jab $a \leq 1$ ya $b \leq 1$ ho — maximum boundary par hota hai, toh endpoints check karo.