Foundations — Properties of estimators — unbiasedness, consistency, efficiency

4.9.18 · D1 · Maths › Probability Theory & Statistics › Properties of estimators — unbiasedness, consistency, effici

Yeh page toolbox hai. Parent note Properties of estimators — unbiasedness, consistency, efficiency padhne se pehle, usmein use hone waale har squiggle ka matlab tumhe samajh aana chahiye. Hum har symbol ko scratch se build karte hain, usse ek picture se anchor karte hain, aur batate hain kyun topic ko uski zaroorat hai. Order matter karta hai: har item pichle wale par lean karta hai.

0. Stage: ek hidden number aur ek random draw

Ek giant barrel imagine karo jisme laakhon numbers hain — yeh population hai. Barrel ke andar kahin, ek true average height hai, ek true spread hai. Yeh true facts fixed hain lekin tumse hidden hain. Tumhe sirf ek mutthi bhar nikaalne ki permission hai.

Neeche ki saari machinery us mutthi (scoop) ko barrel ke baare mein ek honest guess mein badalne ke liye hai.

1. $\theta$ — parameter (hidden number)

Picture: barrel ke andar print ki gayi exact fill-line jo tum kabhi directly nahi padh sakte.

Why the topic needs it: poora subject $\theta$ guess karne ke liye exist karta hai. Bina target ke estimate karne ki koi cheez nahi hai. Har property (unbiased, consistent, efficient) is $\theta$ ke relative measure ki jaati hai.

2. $X_i$ — ek observation, aur "i.i.d."

Capital $X$ (lowercase $x$ nahi) signal karta hai ki yeh random hai — scoop karne se pehle, tumhe nahi pata kya aayega.

Picture: $n$ alag haath usi same barrel mein daakhil ho rahe hain, har haath blindfolded aur doosron se anjaana.

Why the topic needs it: i.i.d. woh assumption hai jo humein expectations ko cleanly add up karne deti hai (Section 6) aur zyada data ko predictably behave karne deti hai (Law of Large Numbers). Ise toro aur unbiasedness/consistency proofs collapse ho jaate hain.

3. $n$ — sample size

Picture: tumhari mutthi mein dots ki sankhya.

Why the topic needs it: $n$ woh dial hai jo tum consistency mein ghumaate ho ("$n\to \infty$ hone par kya hota hai?"). Chhota $n$ = kam dots = jittery guess; bada $n$ = zyada dots = steady guess. Dekhna kaise $n$ baad mein denominators mein appear karta hai (${\sigma }^2/n$, $n-1$) — woh topic ka dil hai.

4. Random variable aur uski distribution

Why the topic needs it: yahan woh plot twist hai jo poore subject ko interesting banata hai — kyunki $\hat{\theta }$ random data se compute hoti hai, $\hat{\theta }$ khud ek random variable hai. Uski apni possible values ki cloud hai, jise sampling distribution kehte hain. Unbiased / consistent / efficient sab us cloud ki shape ke baare mein statements hain.

5. $\mathbb{E}[\cdot ]$ — expectation (long-run average)

Picture: agar distribution ek physical cardboard cut-out hoti, toh $\mathbb{E}[X]$ woh point hai jahan tum ise ek fingertip par balance kar sako.

Why the topic needs it: unbiasedness literally equation $\mathbb{E}[\hat{\theta }]=\theta$ hai ("meri guess-cloud ka balance point truth par baitha hai"). Linearity woh single tool hai jo $\mathbb{E}[\bar{X}]=\mu$ ko do lines mein nikaalta hai.

6. $\sum$ aur $\bar{X}$ — summation aur sample mean

Picture: tumhari mutthi bhar dots ka centre of gravity.

Why the topic needs it: $\bar{X}$ woh starring estimator hai — teeno properties ke liye running example. Notice karo $\bar{X}$ Latin-with-a-bar hai: yeh data se build hoti hai, aur Greek $\mu$ ko estimate karti hai.

7. $\hat{\theta }$ — estimator (hat)

Formally $\hat{\theta }=T(X_1,\dots ,X_n)$ — data ka ek function. Random data daalo, ek (random) number nikalega.

Why the topic needs it: hat poore subject ka protagonist hai. "$\hat{\theta }$ kaisa hai?" woh ek hi sawaal hai jo hum kabhi poochhte hain.

8. $\mathrm{Var}$ aur ${\sigma }^2$ — variance (wobble)

Population variance ${\sigma }^2$ hai (sigma-squared, Greek → truth). Iska square root $\sigma$ standard deviation hai, data ke same units mein.

Why the topic needs it: efficiency smallest variance ka contest hai. Consistency variance $\to 0$ par ride karti hai. Variance teen lenses mein se doosra hai (woh "wobble").

9. $\mathrm{Bias}$ — systematic offset

Picture: ek dartboard. Bias = tumhare average dart ki bullseye se distance. Variance = us average ke around darts kitne scattered hain. Yeh do alag cheezein hain — tumhare paas ek doosre ke bina ho sakta hai.

Why the topic needs it: unbiasedness exactly "bias $=0$" hai, aur MSE (next) bias aur variance dono se built hai.

10. $\mathrm{MSE}$ — honest scorecard

Why the topic needs it: MSE woh single number hai jo tumhe koi bhi do estimators compare karne deta hai — chahe ek biased aur ek unbiased ke beech bhi (parent note ka Uniform example dekho). Yeh easy consistency test bhi deta hai: agar $\mathrm{MSE}\to 0$, estimator consistent hai. Yeh seedha Bias–Variance Tradeoff se link karta hai.

11. Convergence in probability — $\stackrel{P}{\to }$

Symbols unpacked: $|\cdot |$ = distance (absolute value, hamesha $\ge 0$); $\epsilon$ (epsilon) = ek chhota positive "kitna close kaafi close hai"; $P(\cdot )$ = ek event ki probability; ${\lim }_{n\to \infty }$ = "woh value jis par hum settle karte hain jab $n$ bina bound ke badhta hai."

Picture: guess-cloud tighter aur tighter hoti ja rahi hai, target par slide ho rahi hai, jab tak essentially uska saara weight $\theta$ ke around kisi bhi hair's-width band mein nahi aa jaata.

Why the topic needs it: yeh consistency ki exact definition hai. Yeh Law of Large Numbers se power hoti hai, jo guarantee karta hai $\bar{X}\stackrel{P}{\to }\mu$.

12. Likelihood pieces — $f(X;\theta )$, $\ln$, $\partial /\partial \theta$, $I(\theta )$

Yeh sirf efficiency section mein appear karte hain, lekin tumhe inhe pehchanna chahiye.

Why the topic needs it: Fisher Information Cramér–Rao lower bound $\mathrm{Var}(\hat{\theta })\ge 1/I(\theta )$ set karta hai — woh hard floor jise koi bhi unbiased estimator beat nahi kar sakta. Woh floor efficient ki definition hi hai. Yeh Maximum Likelihood Estimation se aur, Central Limit Theorem ke through, isliye link karta hai ki MLEs asymptotically efficient kyun hote hain.

Foundations topic ko kaise feed karte hain

Equipment checklist

Right side cover karo aur parent note kholne se pehle har ek ka jawab do.