Visual walkthrough — Properties of estimators — unbiasedness, consistency, efficiency

4.9.18 · D2 · Maths › Probability Theory & Statistics › Properties of estimators — unbiasedness, consistency, effici

Hum prove karne wale hain:

$$s^2=\frac{1}{n-1}\sum _{i=1}^n(X_i-\bar{X}{)}^2\text{has}\mathbb{E}[s^2]={\sigma }^2.$$

Har symbol use hone se pehle samjhaya jayega.

---

Step 1 — Dots, true centre, aur guessed centre

KYA. $n$ measurements ko ek number line par dots ki tarah imagine karo. Unhe $X_1,\dots ,X_n$ kaho (chota subscript bas dots ko number karta hai: $X_3$ matlab "teesra dot"). Population ka ek hidden true centre hota hai, jise $\mu$ (Greek "mu") likhte hain — iska matlab sirf woh real average hai jo hum dekh nahi sakte. Dots se hum ek guessed centre nikalte hain, yaani sample mean:

$$\bar{X}=\frac{1}{n}\sum _{i=1}^n{X}_i.$$

KYUN. Hum $\mu$ ko formula mein use nahi kar sakte kyunki hume pata nahi. Hamare paas bas $\bar{X}$ hai. "$-1$" ki poori mystery isi substitution se paida hoti hai: hum spread measure karte hain guessed centre ke around, true centre ke nahi.

PICTURE. Line par blue dots, true centre $\mu$ (lavender) aur guessed centre $\bar{X}$ (coral) alag-alag mark kiye — yeh almost kabhi ek jagah nahi hote.

---

Step 2 — "Spread" ka matlab kya hona chahiye: squared distances

KYA. Spread ka matlab hai "dots typically centre se kitni door hain?" Hum har dot ki ek centre se doori maapte hain, use square karte hain, aur average nikalte hain. True population version variance hai:

$${\sigma }^2=\mathbb{E}[(X_i-\mu {)}^2].$$

Square kyun, absolute value kyun nahi? Squaring se algebra cleanly split hoti hai (Step 5 isi par depend karta hai), aur yeh Sampling Distributions mein pehle se defined variance se match karta hai. Yeh door-door ke dots ko zyada punish bhi karta hai, jo "spread" ke feel ke saath sahi baith ta hai.

PICTURE. Har dot ek stick se centre se connected; har stick par ek shaded square — uska area hi squared distance hai.

---

Step 3 — Naive estimator, aur ek shak

KYA. ${\sigma }^2$ ka obvious guess definition ko copy karta hai lekin hamare data aur guessed centre $\bar{X}$ use karta hai:

$${\hat{\sigma }}_{\text{naive}}^2=\frac{1}{n}\sum _{i=1}^n(X_i-\bar{X}{)}^2.$$

Shak kyun? $\bar{X}$ dots ke beech baithne ke liye bana hai — yeh unka balance point hai. Toh $\bar{X}$ se maapi gayi distances average par door-ke true $\mu$ se measured distances se chhoti hoti hain. Naive average bahut chhota niklegaa. Step 4 is hunch ko ek exact equation mein badalta hai.

PICTURE. Wahi dots do baar measure kiye: $\bar{X}$ tak red sticks (chhoti) vs $\mu$ tak lavender sticks (lambi). Red squares clearly chhote dikh rahe hain.

---

Step 4 — Key identity: spread ko split karna

KYA. Hum ek exact bookkeeping identity prove karte hain — abhi koi expectation nahi, bas algebra jo kisi bhi fixed dots ke liye sach hai:

$$\sum _{i=1}^n(X_i-\mu {)}^2=\sum _{i=1}^n(X_i-\bar{X}{)}^2+n(\bar{X}-\mu {)}^2.$$

KYUN yeh poora game hai. Yeh kehta hai: spread-around-true = spread-around-guess + drift-penalty. Rearrange karo, spread-around-guess hai spread-around-true minus ek positive penalty. Yahi Step 3 ka "bahut chhota" hai, ab exact.

PICTURE. Ek balance beam: true-centre spread (left pan) equals guessed-centre spread plus ek choti extra weight $n(\bar{X}-\mu {)}^2$ label ki hui (right pan).

---

Step 5 — Average lo: do facts jo hamare paas hain

KYA. Identity par $\mathbb{E}[\cdot ]$ ("saare samples par average") apply karo. Do ingredients, dono parent note aur LLN ke ideas mein already established hain:

$$\mathbb{E}[(X_i-\mu {)}^2]={\sigma }^2,\mathbb{E}[(\bar{X}-\mu {)}^2]=\mathrm{Var}(\bar{X})=\frac{{\sigma }^2}{n}.$$

Ab Step-4 identity ka poora $\mathbb{E}$ lo:

$$\mathbb{E}[\sum _i(X_i-\bar{X}{)}^2]=\underset{n{\sigma }^2}{\underbrace{\sum _i{\sigma }^2}}-\underset{{\sigma }^2}{\underbrace{n\cdot \frac{{\sigma }^2}{n}}}=(n-1){\sigma }^2.$$

KYUN. Expectation linear hai (yeh sums aur constants se pass ho jaata hai), isliye yahan koi nayi machinery nahi chahiye — sirf upar ke do facts.

PICTURE. Height $n{\sigma }^2$ ki ek bar, jismein se ek ${\sigma }^2$-sized chunk (penalty) kata hua, height $(n-1){\sigma }^2$ ki bar bachti hai.

---

Step 6 — Shrink undo karo: $n-1$ divisor

KYA. Humne paaya ki squared deviations ka sum average karke $(n-1){\sigma }^2$ aata hai. Kuch aisa paane ke liye jo exactly ${\sigma }^2$ par average kare, $n-1$ se divide karo:

$$s^2=\frac{1}{n-1}\sum _{i=1}^n(X_i-\bar{X}{)}^2⟹\mathbb{E}[s^2]=\frac{(n-1){\sigma }^2}{n-1}={\sigma }^2.$$

$n-1$ kyun aur koi aur number kyun nahi? Kyunki Step 5 ne exactly $(n-1)$ produce kiya, $n$ nahi, $n-2$ nahi. Divisor koi fudge nahi hai — yeh woh unique number hai jo average ko sahi banata hai. "$1$" woh single degree of freedom hai jo usi data se centre $\bar{X}$ estimate karne mein kharchi ho gayi.

PICTURE. Do side-by-side dials jo har estimator ka $\mathbb{E}$ dikhate hain: divide-by-$n$ target ${\sigma }^2$ ke neeche point karta hai (biased low), divide-by-$(n-1)$ bilkul target par point karta hai.

---

Step 7 — Edge aur degenerate cases

KYA & KYUN (har case dikhaya gaya taaki koi scenario surprise na kare):

PICTURE. Mini-panels ki ek strip: $n=1$ (akela dot, spread possible nahi), $n=2$ (do dots, ek stick), all-equal (dots stacked, spread $=0$), aur large-$n$ (bias arrow axis ki taraf shrink hoti hui).

---

Ek-picture summary

Upar sab kuch ek sentence mein ek picture ke saath hai: guessed centre $\bar{X}$ se spread measure karna true centre $\mu$ ke bajaye aapko ek ${\sigma }^2$ kharcha kara deta hai, isliye aap $n-1$ effective terms par average karte ho, na ki $n$ par.

Recall Feynman: ek 12-saal ke bachche ko batao

Tum jaanna chahte ho ki dots ka ek group kitna spread out hai. Real spread ka matlab hai "true middle se kitni door." Lekin tumhe true middle pata nahi, toh tum dots ke khud ke average ko stand-in ke roop mein use karte ho. Dikkat yeh hai: dots ka khud ka average hamesha dots ke beech snugly baithta hai, toh usase measured distances thodi chhoti niklati hain — har baar. Kitni chhoti? Humne prove kiya ki yeh exactly ek dot ke worth ki spread hai. Toh total ko saare $n$ dots mein baantne ke bajaye, hum ise $n-1$ mein baantte hain — jaisa ki ek dot "middle pin karne mein use ho gaya." $n-1$ se divide karo aur shortfall perfectly fix ho jaata hai: average par tumhe true spread wapas milti hai. Sirf ek dot ke saath divide karne ke liye kuch nahi ($n-1=0$) — aur sach mein, ek dot spread ke baare mein kuch nahi batata.

Recall Quick self-test

Step 4 mein middle cross-term kyun vanish ho jaata hai? ::: Kyunki ${\sum }_i(X_i-\bar{X})=0$ — dots apne khud ke mean ke around balance karte hain, toh woh factor exactly zero hai. "Penalty term" kya hai aur yeh hamesha positive kyun hai? ::: $n(\bar{X}-\mu {)}^2$, guessed centre ka true centre se squared drift times $n$; ek square kabhi negative nahi hota. Numerically "$-1$" exactly kahaan se aata hai? ::: Penalty $\mathbb{E}[n(\bar{X}-\mu {)}^2]={\sigma }^2$ ko $n{\sigma }^2$ se subtract karne par, $(n-1){\sigma }^2$ bachta hai. Divide-by-$n$ version phir bhi consistent kyun hai? ::: Uska bias $-{\sigma }^2/n$ hai, jo $n\to \infty$ ke saath $\to 0$ ho jaata hai, toh yeh waise bhi ${\sigma }^2$ par converge karta hai.

Related ideas: Bias–Variance Tradeoff (kyun ek unbiased rule automatically best nahi hoti), Maximum Likelihood Estimation (jo aksar biased divide-by-$n$ version produce karta hai), Central Limit Theorem aur Fisher Information (data ${\sigma }^2$ ko kitni tightly pin down kar sakta hai).