Properties of estimators — unbiasedness, consistency, efficiency

4.9.18 · Maths › Probability Theory & Statistics

Estimator KYA hota hai?

Random variable KYU? Kyunki sample lene se pehle, aapko nahi pata kaunsi values aayengi. Alag-alag samples → alag-alag $\hat{\theta }$ values → ek sampling distribution.

1. Unbiasedness

Derivation: $\bar{X}$ kyun $\mu$ ke liye unbiased hai

Maano $X_1,\dots ,X_n$ i.i.d. hain aur $\mathbb{E}[X_i]=\mu$. $\mathbb{E}[\bar{X}]=\mathbb{E}\left[\frac{1}{n}{\sum }_{i=1}^n{X}_i\right]=\frac{1}{n}{\sum }_{i=1}^n\mathbb{E}[X_i]=\frac{1}{n}\cdot n\mu =\mu .$ Yeh step kyun? Expectation linear hai, isliye yeh sum se aur constant $1/n$ se pass ho jaati hai. ✓

Derivation: variance ke liye hum $n-1$ se kyun divide karte hain

Hum ${\sigma }^2=\mathrm{Var}(X)$ estimate karna chahte hain. Naive estimator $\frac{1}{n}\sum (X_i-\bar{X}{)}^2$ biased low hai. Chaliye dekhte hain exactly kyun.

Iss identity se shuru karein (subtract aur add $\mu$): ${\sum }_i(X_i-\bar{X}{)}^2={\sum }_i(X_i-\mu {)}^2-n(\bar{X}-\mu {)}^2.$ Yeh step kyun? $\sum (X_i-\mu {)}^2$ ko $\bar{X}$ ke around expand karo; cross term collapse ho jaata hai kyunki $\sum (X_i-\bar{X})=0$.

Expectations lo: $\mathbb{E}[{\sum }_i(X_i-\bar{X}{)}^2]={\sum }_i\underset{{\sigma }^2}{\underbrace{\mathbb{E}[(X_i-\mu {)}^2]}}-n\underset{\mathrm{Var}(\bar{X})={\sigma }^2/n}{\underbrace{\mathbb{E}[(\bar{X}-\mu {)}^2]}}=n{\sigma }^2-{\sigma }^2=(n-1){\sigma }^2.$ Yeh step kyun? Har $\mathbb{E}[(X_i-\mu {)}^2]={\sigma }^2$ aur $\mathrm{Var}(\bar{X})={\sigma }^2/n$.

Toh ise unbiased banane ke liye hum $n-1$ se divide karte hain, $n$ se nahi:

2. Consistency

Derivation: bias–variance decomposition

Maano $m=\mathbb{E}[\hat{\theta }]$. $\pm m$ insert karo: $\mathbb{E}[(\hat{\theta }-\theta {)}^2]=\mathbb{E}[(\hat{\theta }-m+m-\theta {)}^2].$ Yeh step kyun? Zero jodna $(-m+m)$ hume ek "random part" aur ek "constant part" mein split karne deta hai. Expand karo: $=\underset{\mathrm{Var}}{\underbrace{\mathbb{E}[(\hat{\theta }-m{)}^2]}}+2(m-\theta )\underset{=0}{\underbrace{\mathbb{E}[\hat{\theta }-m]}}+\underset{{\mathrm{Bias}}^2}{\underbrace{(m-\theta {)}^2}}.$ Middle kyun vanish hota hai? $\mathbb{E}[\hat{\theta }-m]=m-m=0$. Toh $\boxed{\text{MSE}=\text{Var}+{\text{Bias}}^2}$.

$\bar{X}$ kyun consistent hai

$\mathrm{Var}(\bar{X})={\sigma }^2/n\to 0$ aur bias $=0$, toh $\mathrm{MSE}={\sigma }^2/n\to 0$. Ho gaya. (Yeh disguise mein Weak Law of Large Numbers hai.)

3. Efficiency

Worked examples

Common mistakes (Steel-man + fix)

Flashcards

Connections

• Maximum Likelihood Estimation — MLEs aksar consistent aur asymptotically efficient hote hain.
• Law of Large Numbers — $\bar{X}$ ki consistency ke neeche yahi hai.
• Central Limit Theorem — $\bar{X}$ ki sampling distribution deta hai, isliye confidence intervals milte hain.
• Fisher Information — variance par CRLB floor define karta hai.
• Bias–Variance Tradeoff — machine learning mein bhi yahi decomposition hai.
• Sampling Distributions — $\hat{\theta }$ ka distribution jise hum judge karte hain.

Concept Map

Deep Dive

• D1 · Foundations — har symbol zero se
• D2 · Visual walkthrough — derivation pictures mein
• D3 · Worked examples — har scenario
• D4 · Exercises — graded, full solutions
• D5 · Question bank — concept traps