Foundations — Central limit theorem

1.3.15 · D1 · AI-ML › Probability & Statistics › Central limit theorem

Yeh page woh har symbol build karta hai jo parent note use karta hai, shuru karta hai un cheezon se jo ek curious 12-saal-ka bachcha already jaanta hai: dice, coin flips, aur dot-patterns. Hum kuch bhi add nahi karte jab tak aap us cheez ko dekh nahi lete.

1. Randomness: random variable $X$

Ise picture karo. Ek die roll karo. Dekhne se pehle, result unknown hai — lekin woh $\{1,2,3,4,5,6\}$ mein se kuch ek hai. Yeh poora "machine jo ek unknown number produce karta hai" woh $X$ hai.

Topic ko iska kyun zaroorat hai. Machine learning mein har piece of data — ek pixel value, ek gradient, ek click/no-click — ek aisa number hai jo kisi random process se aaya hai. Hume ek letter chahiye jisse "ek single draw" ke baare mein baat kar sakein, kaafi draws ki baat karne se pehle.

Jab hamare paas kaafi draws hote hain toh hum unhe number karte hain: $X_1,X_2,\dots ,X_n$. Subscript sirf ek label hai ("draw number 1", "draw number 2"), aur $n$ hai kitne draws humne liye.

2. Distribution: randomness ki shape

Ise picture karo. Ek fair die ke liye har face ki same chance hai $1/6$, toh uski distribution flat hai — chhe equal-height bars. Hum ise uniform kehte hain kyunki har value equally likely hai.

Topic ko iska kyun zaroorat hai. CLT ka poora magic yeh hai ki use parwah nahi ki starting shape kya hai — flat, lopsided, spiky, discrete ya continuous — averages phir bhi bell-shaped end up hote hain. Yeh appreciate karne ke liye, aapko pehle starting shape dekhne ki zaroorat hai.

3. Mean $\mu$ aur symbol $\mathbb{E}[\cdot ]$

Ise picture karo. Har possible value ko ek see-saw par mark karo aur har ek par uski probability ke barabar ek weight rakho. Jis point par see-saw balance kare woh $\mu$ hai. Die ke liye, balance point bilkul beech mein $3.5$ par hai.

$\mu =\mathbb{E}[X]=\frac{1+2+3+4+5+6}{6}=3.5$

Topic ko iska kyun zaroorat hai. $\mu$ woh sahi jawab hai jise hum data se estimate karne ki koshish kar rahe hain. CLT ek statement hai ki samples ka average is $\mu$ ke kitna karib aata hai.

4. Spread: variance $\text{Var}(X)={\sigma }^2$ aur standard deviation $\sigma$

Ise picture karo. Do dartboards, dono $\mu$ par centred. Ek mein darts tight cluster mein hain (chota $\sigma$), ek mein darts wide scatter hain (bada $\sigma$). Distances ko square karna matlab ek dart jo double distance par hai woh char guna count karta hai, isliye outliers dominate karte hain.

Topic ko iska kyun zaroorat hai. $\sigma$ measure karta hai ki ek single measurement kitna noisy hai. CLT ka headline result yeh hai ki averaging is spread ko ek precise way mein shrink karta hai — toh pehle hume spread ka naam pata hona chahiye.

5. Independence aur "identically distributed" (i.i.d.)

Ise picture karo. Ek factory socho jo dice print kar rahi hai, sab identical (identically distributed), aur aap unhe separate rooms mein roll karte ho toh koi bhi roll doosre ko affect nahi kar sakta (independent).

Topic ko iska kyun zaroorat hai. Independence woh cheez hai jo humein variances add karne deti hai ($\text{Var}(X+Y)=\text{Var}(X)+\text{Var}(Y)$) — yeh key algebra step hai jo hum bilkul agले section mein aur phir CLT proof mein use karte hain. "Identically distributed" woh cheez hai jo humein yeh kehne deti hai ki har draw ka same $\mu$ aur ${\sigma }^2$ hai.

6. Sample mean ${\bar{X}}_n$

Notation padhna. Symbol ${\sum }_{i=1}^n{X}_i$ sirf shorthand hai "$X_1+X_2+\cdots +X_n$ add karo" ke liye. $\sum$ capital Greek S hai ("Sigma", Sum ke liye); $i$ counter hai jo $1$ se $n$ tak chalta hai. Phir $\frac{1}{n}$ divide karta hai kitne add kiye — yeh ek average hai.

Ise picture karo. Jab bhi aap ek average compute karte ho, aapko ek naya number milta hai. Poora experiment dobara karo aur thoda alag average milega. Toh ${\bar{X}}_n$ khud ek random variable hai apni distribution ke saath — aur CLT exactly woh distribution describe karta hai.

Topic ko iska kyun zaroorat hai. ${\bar{X}}_n$ hi show ka star hai. CLT ek statement hai ki $n$ badhne par ${\bar{X}}_n$ ka kya hota hai.

7. Standard error: $\sigma /\sqrt{n}$

Ise picture karo. Jaise jaise aap zyada numbers average karte ho, lucky-high aur unlucky-low draws cancel out ho jaate hain, toh average kam wobble karta hai. Denominator mein $\sqrt{n}$ kehta hai: wobble half karne ke liye aapko chaar guna samples chahiye (kyunki $\sqrt{4}=2$).

Topic ko iska kyun zaroorat hai. Yeh CLT ka quantitative punch-line hai: averaging sirf cheezein bell-shaped nahi banata, balki bell ko $1/\sqrt{n}$ ki rate se narrow bhi karta hai. Yeh ek fact the law of large numbers, Confidence Intervals, aur Sample Size Calculation ko power karta hai.

8. Standardizing: $Z$-score

Ise picture karo. Do bell curves alag-alag widths ki alag-alag jagah baithi hain. Standardizing har ek ko slide karta hai taki uska centre $0$ par aa jaaye, phir stretch ya squeeze karta hai taki uski width exactly $1$ ho jaaye. Ab woh perfectly overlap karti hain — ek fair comparison.

Topic ko iska kyun zaroorat hai. CLT sample mean ko ek single reference curve se compare karta hai, standard normal. Woh comparison karne ke liye hum pehle ${\bar{X}}_n$ standardize karte hain: $Z_n=\frac{{\bar{X}}_n-\mu }{\sigma /\sqrt{n}}.$ Numerator: "hamaari average true value $\mu$ se kitni door hai." Denominator: standard error. Toh $Z_n$ padhta hai "hum kitne standard errors door hain?"

9. Normal distribution $\mathcal{N}(\mu ,{\sigma }^2)$ aur $\Phi$

Ise picture karo. Ek single humped curve, centre par sabse tall, dono taraf symmetrically tail off karti hui. Poori picture Normal Distribution mein hai.

Topic ko iska kyun zaroorat hai. Jab CLT humein bata de ki $Z_n$ approximately standard normal hai, toh sample mean ke baare mein koi bhi probability question is ek fixed curve ke neeche ek area ban jaata hai — $\Phi$ ke through padha jaata hai. Yeh exactly wahi hai jo parent ke worked examples karte hain.

10. "Converges in distribution" arrow $\stackrel{d}{\to }$

Ise picture karo. Histograms ki ek sequence, har ek bade $n$ ke liye, frame by frame morph hoti hai jab tak woh bell curve ke upar bilkul fit na ho jaaye.

Topic ko iska kyun zaroorat hai. CLT ka core statement hai $Z_n\stackrel{d}{\to }\mathcal{N}(0,1)$: standardized sample mean ki shape standard normal ban jaati hai.

11. Fine print: finite variance $\mathbb{E}[X^2]<\infty$

Ise picture karo. Ek "normal" distribution ke tails fast shrink hote hain, toh extreme values itne rare hain ki squared distances finite rehti hain. Ek pathological heavy-tailed distribution giant outliers produce karta rehta hai jinke squares har average dominate karte hain — bell curve kabhi nahi banti.

Topic ko iska kyun zaroorat hai. Parent ki definition likhti hai "$\text{Var}(X_i)={\sigma }^2<\infty$" — woh chota sa "$<\infty$" yahi assumption hai. Yeh woh ek condition hai jo "averaging tames randomness" aur "averaging can't help you" ke beech khadi hai.

Yeh theorem ko kaise feed karte hain

Equipment checklist

Right side cover karo aur khud ko test karo. Agar koi bhi answer fuzzy lage, toh main proof se pehle woh section dobara padho.

Taiyaar ho? Central limit theorem par jaao poori derivation ke liye, aur Law of Large Numbers dekho uske close cousin ke liye.