Worked examples — Properties of estimators — unbiasedness, consistency, efficiency

4.9.18 · D3 · Maths › Probability Theory & Statistics › Properties of estimators — unbiasedness, consistency, effici

Yeh parent topic ka ek companion drill page hai. Parent ne teen lenses banaye the — unbiasedness (average pe sahi), consistency (jaise $n$ badhta hai, paas aata hai), aur efficiency (sabse kam wobbly). Yahaan hum har tarah ka scenario ek-ek worked example ke saath cover karte hain.

Shuru karne se pehle, chaar symbols jo baar baar use honge. Agar koi line inhe use kare, toh yahaan se earn kiye gaye hain:

---

Scenario matrix

Har estimator question in cells mein se kisi ek mein hoti hai. Neeche ke examples us cell ke label se marked hain, aur milke puri grid cover karte hain — Ex 1 se Ex 10 tak C1 se C10 tak, kuch bhi skip nahi.

Cell	Case class	Kya galat ho sakta hai / test ho sakta hai	Example
C1	Fixed $n$ pe Unbiased (linear estimator)	kya weights ka sum 1 hai?	Ex 1
C2	Efficiency comparison (dono unbiased)	kaun sa variance chhota hai, kitne se	Ex 2
C3	Optimal weighting (unequal variances)	naive equal weights best nahi hote	Ex 3
C4	Degenerate estimator (ek point use karta hai)	unbiased lekin NOT consistent	Ex 4
C5	Biased estimator, boundary/max statistic	bias ka sign, unbiased correction	Ex 5
C6	MSE unbiasedness ko beat karta hai (shrinkage)	biased MSE pe jeet sakta hai	Ex 6
C7	Limiting behaviour ($n\to \infty$)	biased rule ki consistency	Ex 7
C8	Cramér–Rao efficiency (kya bound hit karta hai?)	Var vs $1/(n{I}_1)$	Ex 8
C9	Real-world word problem	story → estimator translate karo	Ex 9
C10	Exam twist (do rules combine karo)	best linear combination	Ex 10

Prerequisite links jo padhte waqt kholna chahoge: Sampling Distributions, Law of Large Numbers, Fisher Information, Bias–Variance Tradeoff, Maximum Likelihood Estimation, Central Limit Theorem.

---

Example 1 — C1: kya ek weighted average unbiased hai?

Forecast: abhi andaza lagao — kya yeh average pe $\mu$ hit karta hai, ya tilted hai? (Hint: weights dekho.)

1. Expectation likho aur linearity use karo. $\mathbb{E}[\hat{\mu }]=0.5\mathbb{E}[X_1]+0.2\mathbb{E}[X_2]+0.3\mathbb{E}[X_3]=(0.5+0.2+0.3)\mu .$ Yeh step kyun? Expectation sums aur constants ke through pass hoti hai (linearity), isliye data ka weighted average means ka same weighted average ban jaata hai. Har $\mathbb{E}[X_i]=\mu$.

2. Weights add karo. $0.5+0.2+0.3=1\Rightarrow \mathbb{E}[\hat{\mu }]=1\cdot \mu =\mu .$ Yeh step kyun? Bias hai $(\text{sum of weights}-1)\mu$. Yeh exactly tab vanish hota hai jab weights ka sum 1 ho — yeh kisi bhi linear estimator of the mean ke liye general rule hai.

Verify: agar weights $0.5,0.2,0.4$ hote (sum $1.1$) toh $\mathbb{E}[\hat{\mu }]=1.1\mu \ne \mu$ — biased. Hamare weights ka sum 1 hai, isliye unbiased. ✓

---

Example 2 — C2: kaun sa unbiased estimator zyada efficient hai?

Forecast: kaun sa cloud tighter hai — teeno points use karne wala, ya $X_2$ throw kar dene wala?

1. ${\hat{\mu }}_A$ ka Variance. Independent variables ke liye, weighted sum ka variance $\sum w_i^2{\sigma }^2$ hai. $\mathrm{Var}({\hat{\mu }}_A)={\left(\frac{1}{3}\right)}^2{\sigma }^2\cdot 3=\frac{{\sigma }^2}{3}.$ Yeh step kyun? Independence saare covariance cross-terms khatam kar deta hai, isliye sirf $w_i^2$ terms bachte hain.

2. ${\hat{\mu }}_B$ ka Variance. $\mathrm{Var}({\hat{\mu }}_B)={\left(\frac{1}{2}\right)}^2{\sigma }^2\cdot 2=\frac{{\sigma }^2}{2}.$ Yeh step kyun? Same rule, lekin ab sirf do points hain jinmein se har ek weight $\frac{1}{2}$ carry karta hai, isliye do $w_i^2$ terms bachte hain.

3. Relative efficiency.

Verify: ${\sigma }^2/3<{\sigma }^2/2$ kisi bhi ${\sigma }^2>0$ ke liye, isliye ${\hat{\mu }}_A$ more efficient hai. Saara data use karna payoff deta hai. ✓

---

Example 3 — C3: jab variances alag hon toh optimal weights

Forecast: kya tumhe noisy $Y_2$ ko $Y_1$ ke equal, kam, ya zyada weight dena chahiye?

1. Unbiasedness weights ko 1 sum karne pe fix karta hai. Weights $w$ aur $1-w$ ke saath, $\mathbb{E}[\hat{\mu }]=\mu$ automatically aata hai. Accha — ek constraint handle hua. Yeh step kyun? Ex 1 ke rule se, weights ka sum $1$ hone se unbiasedness guarantee hoti hai; yeh doosre weight ko $1-w$ pin karta hai aur ek free parameter $w$ optimise karne ke liye chodta hai.

2. Variance likho. $V(w)=w^2\cdot 1+(1-w{)}^2\cdot 4.$ Yeh step kyun? Phir se independence → do scaled variances add karo.

3. Minimise karo: differentiate karo aur zero set karo. $V^{\prime }(w)=2w-8(1-w)=10w-8=0\Rightarrow w=\frac{8}{10}=\mathrm{0.8.}$ Derivative kyun? $V(w)$ $w$ mein ek upward parabola hai; iska sabse neeche wala point wahan hai jahan slope zero hai. Wahi $V^{\prime }(w)=0$ locate karta hai. General rule: har measurement ko apne variance ke inversely proportional weight do — yahaan $\frac{1/1}{1/1+1/4}=\frac{1}{1.25}=0.8$. ✓ match karta hai.

4. Resulting minimum variance. $V(0.8)=0.8^2+0.2^2\cdot 4=0.64+0.16=\mathrm{0.8.}$ Yeh step kyun? Optimal $w$ ko $V(w)$ mein plug karo taaki sabse chhota achievable variance padh sako.

Verify: equal weights $w=0.5$ dete hain $V=0.25+0.25\cdot 4=1.25>0.8$. Inverse-variance weighting jeet jaata hai. Lesson: equal weights (Ex 2) optimal hote hain sirf jab variances equal hon. ✓

---

Example 4 — C4: unbiased lekin NOT consistent (degenerate rule)

Forecast: yeh obviously "silly" hai, lekin dono properties mein se actually kaun si fail hoti hai?

1. Unbiased? $\mathbb{E}[X_1]=\mu$. Haan, har $n$ ke liye unbiased. Yeh step kyun? Pehla draw mean $\mu$ rakhta hai chahe kitne bhi aur points hon.

2. Consistency check via MSE. Bias $=0$, isliye $\mathrm{MSE}=\mathrm{Var}(X_1)={\sigma }^2$ — ek constant, $n$ se independent. Yeh step kyun? Consistency ke liye $\mathrm{MSE}\to 0$ chahiye. Yahaan yeh kabhi shrink nahi hota; zyada data collect karna kuch nahi karta kyunki tum use throw away kar dete ho.

3. Conclusion. $\mathrm{MSE}={\sigma }^2\nrightarrow 0\Rightarrow$ not consistent. Yeh step kyun? MSE→0 ka sufficient condition constant MSE ke liye kabhi fire nahi kar sakta, isliye estimator limit test fail karta hai.

Verify: $\bar{X}$ se compare karo jahaan $\mathrm{MSE}={\sigma }^2/n\to 0$. Same bias (zero), bilkul alag limit. Yeh concrete counterexample hai "unbiased $\Rightarrow$ consistent" ka. ✓

---

Example 5 — C5: boundary/max statistic — bias ka sign

Forecast: maximum kabhi $\theta$ se exceed nahi kar sakta — kya yeh estimate ko truth ke upar ya neeche kheechta hai?

Figure s01 — size $n=3$ ka ek Uniform$(0,\theta )$ sample value axis pe: teeno points aur unka maximum strictly true upper limit $\theta$ ke neeche baithe hain, aur right pe plum gap downward bias hai.

1. Max ka Distribution. $P(\max \le t)={\left(\frac{t}{\theta }\right)}^n$ for $0\le t\le \theta$. Yeh step kyun? Saare $n$ points $t$ ke neeche fall karne chahiye; har ek aisa $t/\theta$ probability se karta hai, aur woh independent hain, isliye hum multiply karte hain.

2. Max ka Expectation. Integrate karne pe standard result milta hai $\mathbb{E}[\max ]=\frac{n}{n+1}\theta .$ Yeh step kyun? Density hai $\frac{n{t}^{n-1}}{{\theta }^n}$; ${\int }_0^{\theta }t\cdot \frac{n{t}^{n-1}}{{\theta }^n}dt=\frac{n}{n+1}\theta$.

3. Bias ka sign. $\mathrm{Bias}=\frac{n}{n+1}\theta -\theta =-\frac{1}{n+1}\theta <0$. Maximum hamesha underestimate karta hai — figure se match karta hai, jahaan maximum $[0,\theta ]$ ke andar baithta hai aur sirf $\theta$ ke paas se aa sakta hai, kabhi reach nahi kar sakta. Yeh step kyun? Bias har $n$ ke liye negative hai, isliye yeh ek systematic under-shoot hai, random luck nahi.

4. Unbiased fix. $\frac{n+1}{n}$ se multiply karo: ${\hat{\theta }}_{\text{unb}}=\frac{n+1}{n}{\max }_i{X}_i,\mathbb{E}[{\hat{\theta }}_{\text{unb}}]=\theta .$ Yeh step kyun? $\frac{n}{n+1}$ ke reciprocal se exactly scale up karna shrink cancel kar deta hai, $\mathbb{E}=\theta$ restore karta hai.

Verify (n=3): $\mathbb{E}[\max ]=\frac{3}{4}\theta$, bias $=-\frac{1}{4}\theta$. Correction factor $\frac{4}{3}$, aur $\frac{4}{3}\cdot \frac{3}{4}\theta =\theta$. ✓

---

Example 6 — C6: ek biased estimator jo unbiased wale ko MSE pe beat karta hai

Forecast: unbiased sunne mein best lagta hai — lekin agar hum estimate thoda shrink karein, kya thodi si bias aur kam variance trade karna total MSE kam kar sakti hai?

Hume $n=3,\theta =1$ ke liye $M$ ke baare mein do facts chahiye: $\mathbb{E}[M]=\frac{3}{4}$ aur $\mathbb{E}[M^2]=\frac{n}{n+2}=\frac{3}{5}$, isliye $\mathrm{Var}(M)=\frac{3}{5}-\frac{9}{16}=\frac{3}{80}$.

1. General ${\hat{\theta }}_c=cM$ ka MSE. Bias $c\mathbb{E}[M]-1=\frac{3}{4}c-1$ aur variance $c^2\mathrm{Var}(M)$ ke saath, $\mathrm{MSE}(c)=c^2\mathbb{E}[M^2]-2c\mathbb{E}[M]+1=\frac{3}{5}{c}^2-\frac{3}{2}c+1.$ Yeh step kyun? $\mathrm{MSE}=\mathbb{E}[(cM-1{)}^2]=c^2\mathbb{E}[M^2]-2c\mathbb{E}[M]+1$ — bas square expand karo; yeh Var aur Bias${}^2$ ko $c$ mein ek clean quadratic mein bundle kar deta hai.

2. $c$ pe minimise karo. Differentiate karo aur zero set karo: ${\mathrm{MSE}}^{\prime }(c)=\frac{6}{5}c-\frac{3}{2}=0\Rightarrow c^{\star }=\frac{3/2}{6/5}=\frac{5}{4}=\mathrm{1.25.}$ Derivative kyun? $\mathrm{MSE}(c)$ $c$ mein ek upward parabola hai; iska minimum wahan hai jahan slope zero hai. MSE-optimal factor $c^{\star }=\frac{5}{4}$ unbiased factor $\frac{4}{3}\approx 1.33$ se chhota hai — yeh deliberately estimate shrink karta hai, thodi bias accept karta hai.

3. Optimal (biased) rule pe MSE. $\mathrm{MSE}(\frac{5}{4})=\frac{3}{5}\cdot \frac{25}{16}-\frac{3}{2}\cdot \frac{5}{4}+1=\frac{15}{16}-\frac{15}{8}+1=\frac{1}{16}=\mathrm{0.0625.}$ Yeh step kyun? $c^{\star }$ ko quadratic mein plug karo taaki is family mein sabse chhota attainable MSE padh sako.

4. Unbiased $\frac{4}{3}M$ ka MSE. Bias $=0$, isliye $\mathrm{MSE}(\frac{4}{3})={\left(\frac{4}{3}\right)}^2\mathrm{Var}(M)=\frac{16}{9}\cdot \frac{3}{80}=\frac{1}{15}\approx \mathrm{0.0667.}$ Yeh step kyun? Jab bias zero ho toh MSE sirf variance hota hai, isliye $\mathrm{Var}(M)$ ko squared factor se scale karo.

5. Compare karo. $0.0625<0.0667$: biased shrunk rule $c^{\star }=\frac{5}{4}$ ka MSE unbiased rule se strictly kam hai. Yeh kyun dikhao? Yahi Bias–Variance Tradeoff ka honest headline hai: "unbiased" aur "best" same nahi hain. Thodi deliberate bias kaafi variance reduction khareed sakti hai jo total squared error kam kar de.

Verify: teeno MSE numbers ($c=\frac{5}{4}\to \frac{1}{16}$, $c=\frac{4}{3}\to \frac{1}{15}$, aur $\frac{1}{16}<\frac{1}{15}$) neeche check kiye. ✓

---

Example 7 — C7: ek biased rule jo phir bhi consistent hai

Pehle, do objects jo yeh example lean karta hai — use se pehle define karo:

Forecast: har finite $n$ pe biased — kya bias limit mein survive karta hai, ya wash out ho jaata hai?

1. Bias. Parent se, $\mathbb{E}[\sum (X_i-\bar{X}{)}^2]=(n-1){\sigma }^2$. $n$ se divide karne pe: $\mathbb{E}[{\hat{\sigma }}_n^2]=\frac{n-1}{n}{\sigma }^2,\mathrm{Bias}=\frac{n-1}{n}{\sigma }^2-{\sigma }^2=-\frac{{\sigma }^2}{n}.$ Yeh step kyun? $\mu$ ki jagah $\bar{X}$ pe center karne se deviations shrink hoti hain, ek degree of freedom costing hoti hai — classic $n-1$ story.

2. Variance bhi shrink hota hai. ${\hat{\sigma }}_n^2=\frac{n-1}{n}{s}^2$ likho jahaan $s^2$ abhi define kiya gaya unbiased sample variance hai. Tab $\mathrm{Var}({\hat{\sigma }}_n^2)={\left(\frac{n-1}{n}\right)}^2\mathrm{Var}(s^2),\mathrm{Var}(s^2)=\frac{1}{n}\left({\mu }_4-\frac{n-3}{n-1}{\sigma }^4\right).$ Yeh step kyun? $\mathrm{Var}(s^2)$ ke standard formula mein explicitly $\frac{1}{n}$ bahar hai (iske liye finite fourth central moment ${\mu }_4$ chahiye). Isliye $\mathrm{Var}(s^2)\to 0$ $1/n$ ki tarah, aur ${\left(\frac{n-1}{n}\right)}^2\to 1$ se multiply karne pe $\mathrm{Var}({\hat{\sigma }}_n^2)\to 0$ bhi milta hai. MSE ke dono pieces vanish ho jaate hain.

3. Bias ki limit. $-{\sigma }^2/n\to 0$ as $n\to \infty$. Yeh step kyun? Ab hamare paas $\mathrm{MSE}=\mathrm{Var}+{\mathrm{Bias}}^2$ ke dono ingredients zero jaate hue hain.

4. Conclusion: MSE → 0 ⇒ consistent. Kyunki $\mathrm{Var}({\hat{\sigma }}_n^2)\to 0$ (step 2) aur ${\mathrm{Bias}}^2={\sigma }^4/n^2\to 0$ (step 3), $\mathrm{MSE}({\hat{\sigma }}_n^2)=\mathrm{Var}+{\mathrm{Bias}}^2\to 0\Rightarrow {\hat{\sigma }}_n^2\text{ is }\mathbf{consistent}.$ Yeh step kyun? MSE→0 ka sufficient condition fire karta hai: chahe estimator har finite $n$ pe biased ho, bias limit mein vanish ho jaata hai, isliye consistency hold karti hai.

Verify (n=10, σ²=5): $\mathbb{E}[{\hat{\sigma }}_{10}^2]=\frac{9}{10}\cdot 5=4.5$, bias $=-0.5$. Jaise $n\to \infty$, factor $\frac{n-1}{n}\to 1$ aur $\mathrm{Var}\to 0$. ✓ Biased ≠ inconsistent.

---

Example 8 — C8: kya ek estimator Cramér–Rao bound hit karta hai?

Forecast: hum pehle se jaante hain $\mathrm{Var}(\bar{X})={\sigma }^2/n$. Kya information bound exactly wahan land karega, ya improve karne ki room bachegi?

1. Fisher information per observation. Log-density hai $\ln f=-\frac{(x-\mu {)}^2}{2{\sigma }^2}+\text{const}$. Iska $\mu$-derivative (score): $\frac{\partial }{\partial \mu }\ln f=\frac{x-\mu }{{\sigma }^2}.$ Yeh tool kyun? Fisher Information expected squared score hai — yeh measure karta hai ki likelihood $\mu$ pe kitna sharply react karti hai. Sharp reaction matlab data $\mu$ ko pin down kar raha hai, isliye achievable variance chhota hai.

2. Square karo aur expectation lo. $I_1(\mu )=\mathbb{E}\left[{\left(\frac{X-\mu }{{\sigma }^2}\right)}^2\right]=\frac{\mathbb{E}[(X-\mu {)}^2]}{{\sigma }^4}=\frac{{\sigma }^2}{{\sigma }^4}=\frac{1}{{\sigma }^2}.$ Yeh step kyun? $\mathbb{E}[(X-\mu {)}^2]={\sigma }^2$ variance ki definition se, isliye ratio $1/{\sigma }^2$ pe collapse ho jaata hai.

3. $n$ i.i.d. points ke liye CRLB. $\mathrm{Var}(\hat{\mu })\ge \frac{1}{n{I}_1(\mu )}=\frac{1}{n\cdot (1/{\sigma }^2)}=\frac{{\sigma }^2}{n}.$ Yeh step kyun? Information independent observations ke across add hoti hai, isliye total information $n{I}_1$ hai, aur CRLB iska reciprocal hai.

4. $\bar{X}$ se compare karo. $\mathrm{Var}(\bar{X})={\sigma }^2/n$ = bound exactly. Yeh step kyun? CRLB ke saath equality efficient estimator ki definition hai.

Verify: bound $={\sigma }^2/n=$ actual variance $\Rightarrow$ $\bar{X}$ efficient hai (known-variance normal mean ke liye MVUE). ✓

---

Example 9 — C9: real-world word problem

Forecast: andaza lagao ki $256$ bolts measure karna $64$ se kitna zyada precise hai — double? quadruple?

1. (a) Properties. $\bar{X}$ unbiased hai ($\mathbb{E}[\bar{X}]=\mu$), consistent hai ($\mathrm{Var}={\sigma }^2/n\to 0$), aur normal mean ke liye efficient hai (Ex 8). Isliye yeh best simple choice hai, aur reported estimate $50.1$ mm hai. Yeh step kyun? Hum story ko estimator $\bar{X}$ mein translate karte hain aur number trust karne se pehle teeno lenses check karte hain.

2. (b) Standard error at $n=64$. Standard error hai $\mathrm{SD}(\bar{X})=\sigma /\sqrt{n}$. $\frac{2}{\sqrt{64}}=\frac{2}{8}=0.25\text{ mm}.$ Yeh step kyun? Variance $1/n$ ki tarah girta hai, isliye standard deviation (iska square root) $1/\sqrt{n}$ ki tarah girta hai. Units: mm, $\mu$ se match karta hai.

3. Standard error at $n=256$. $\frac{2}{\sqrt{256}}=\frac{2}{16}=0.125\text{ mm}.$ Yeh step kyun? Same formula $\sigma /\sqrt{n}$ new sample size ke saath; bada sample chhota number deta hai, yane tighter estimate.

4. Kitna shrink hua? Dono standard errors ka ratio lo: $\frac{\text{SE}(64)}{\text{SE}(256)}=\frac{0.25}{0.125}=2.$ Yeh step kyun? $n$ ko quadruple karna (from $64$ to $256$) $\sqrt{n}$ ko $\sqrt{4}=2$ se multiply karta hai, isliye standard error half ho jaata hai — quartered nahi. Precision sirf $\sqrt{n}$ ki tarah improve hoti hai, woh diminishing return jo har mean estimator mein baka hua hai (yeh Law of Large Numbers hai jiska rate Central Limit Theorem set karta hai).

Verify: SE at $64=0.25$ mm, at $256=0.125$ mm, ratio $=2$. ✓ Units mm throughout, aur dono values positive hain aur expected ki tarah shrink kar rahi hain.

---

Example 10 — C10: exam twist — do estimators ka best combination

Forecast: kyunki ${\hat{\mu }}_1$ pehle se tighter hai, kya best $w$ $0.5$ se upar hoga? Aur kya combining ${\hat{\mu }}_1$ alone ko beat kar sakta hai?

1. Inverse-variance weighting. Same machinery jaise Ex 3: optimal $w=\frac{1/2}{1/2+1/6}$. $w=\frac{1/2}{1/2+1/6}=\frac{1/2}{2/3}=\frac{3}{4}=\mathrm{0.75.}$ Yeh step kyun? $w^2(2)+(1-w{)}^2(6)$ ko derivative se minimise karne pe $w=\frac{V_2}{V_1+V_2}=\frac{6}{8}=0.75$ milta hai — equivalently inverse-variance weights.

2. Minimum variance. $V_{\min }=\frac{1}{1/V_1+1/V_2}=\frac{1}{1/2+1/6}=\frac{1}{2/3}=\mathrm{1.5.}$ Yeh step kyun? Independent unbiased estimators ke inverse-variance weighting ke liye, combined precision (reciprocal variance) individual precisions ka sum hota hai.

3. ${\hat{\mu }}_1$ alone ke versus Relative efficiency (better over worse). $\frac{\mathrm{Var}({\hat{\mu }}_1)}{V_{\min }}=\frac{2}{1.5}=\frac{4}{3}\approx \mathrm{1.33.}$ Yeh step kyun? Ek noisier estimator ko bhi combine karna variance strictly lower karta hai ($2$ se $1.5$ tak), isliye combination tighter hai; iska variance denominator mein rakhne se ratio $>1$ aata hai.

Verify: $w=0.75$ plug karo: $0.75^2\cdot 2+0.25^2\cdot 6=1.125+0.375=1.5$ ✓, closed form se match karta hai. Relative efficiency $4/3$. ✓

---

Recall Quick self-test (answers cover karo)

Unbiased but not consistent — rule ka naam batao ::: $\hat{\mu }=X_1$ (variance kabhi shrink nahi hoti). Independent estimators ke liye optimal weight proportional hota hai ::: har variance ke reciprocal ke (inverse-variance weighting). $\bar{X}$ ka standard error scale hota hai ::: $1/\sqrt{n}$ ki tarah — $n$ ko quadruple karo halve karne ke liye. $\bar{X}$ normal mean ke liye Cramér–Rao bound hit karta hai kyunki ::: iska variance ${\sigma }^2/n$, $1/(n{I}_1)$ ke barabar hai jahan $I_1=1/{\sigma }^2$. Uniform$(0,\theta )$ sample ka max biased hai ::: neeche (hamesha underestimate karta hai), factor $\frac{n+1}{n}$ se fix hota hai; MSE-optimal shrinkage factor aur bhi chhota hai, $n=3$ pe $\frac{5}{4}$.