Central limit theorem — Hinglish notes

1.3.15 · AI-ML › Probability & Statistics

Definition

HAR piece ka matlab kya hai?

i.i.d.: Har sample independently same distribution se draw kiya gaya hai
$μ$ : Woh true population mean jo hum estimate kar rahe hain
$σ^{2}$ : True population variance (individual data points ka spread)
$σ / n$ : Mean ka Standard error—sample means $μ$ ke aas-paas kitna vary karte hain

Derivation from First Principles

GOAL: Prove karo ki standardized sample means $N (0, 1)$ ki taraf approach karte hain.

Step 1: Standardize Kyun Karein?

Hum alag-alag scales ki distributions compare karna chahte hain. Standardization kisi bhi random variable ko zero mean aur unit variance mein transform karta hai: $Z = \frac{X - E [ X ]}{Var ( X )}$

YEH transformation kyun? Yeh comparison "fair" banata hai—sab variables ab same scale par hain.

Sample mean ke liye:

$E [\overset{ˉ}{X}_{n}] = E [\frac{1}{n} \sum_{i = 1}^{n} X_{i}] = \frac{1}{n} \sum_{i = 1}^{n} E [X_{i}] = \frac{1}{n} (n μ) = μ$ ✓
$Var (\overset{ˉ}{X}_{n}) = Var (\frac{1}{n} \sum_{i = 1}^{n} X_{i})$

$1/ n^{2}$ kyun bahar nikal sakte hain? Kyunki $Var (c X) = c^{2} Var (X)$ : $= \frac{1}{n ^{2}} Var (\sum_{i = 1}^{n} X_{i})$

Variances sum kyun kar sakte hain? Independence: $Var (X + Y) = Var (X) + Var (Y)$ jab $X ⊥ Y$ : $= \frac{1}{n ^{2}} \sum_{i = 1}^{n} Var (X_{i}) = \frac{1}{n ^{2}} (n σ^{2}) = \frac{σ ^{2}}{n}$

Therefore: $\overset{ˉ}{X}_{n}$ ka mean $μ$ hai aur variance $σ^{2} / n$ hai.

Step 2: Moment Generating Functions (The Proof Engine)

Ek random variable $Y$ ka moment generating function (MGF) hai: $M_{Y} (t) = E [e^{t Y}]$

MGFs kyun use karein? Key theorem: Agar $M_{Y_{n}} (t) \to M_{Y} (t)$ sab $t$ ke liye, toh $Y_{n} d Y$ . MGFs distributions ko uniquely characterize karte hain.

Standard normal ke liye: $M_{N (0, 1)} (t) = e^{t^{2} /2}$

Step 3: Standardized Sample Mean ka MGF

Standardized variables define karo: $Y_{i} = \frac{X _{i} - μ}{σ}$

Toh $E [Y_{i}] = 0$ , $Var (Y_{i}) = 1$ , aur: $Z_{n} = \frac{X ˉ _{n} - μ}{σ / n} = \frac{\sum _{i = 1}^{n} ( X _{i} - μ )}{n \cdot σ / n} = \frac{1}{n} \sum_{i = 1}^{n} Y_{i}$

YEH form kyun? Humne problem ko standardized variables tak reduce kar diya hai.

MGF hai: $M_{Z_{n}} (t) = E [exp (\frac{t}{n} \sum_{i = 1}^{n} Y_{i})]$

Product kyun split kar sakte hain? Independence + exponential property $e^{a + b} = e^{a} \cdot e^{b}$ : $= E [\prod_{i = 1}^{n} exp (\frac{t}{n} Y_{i})] = \prod_{i = 1}^{n} E [exp (\frac{t}{n} Y_{i})]$

Kyunki $Y_{i}$ identically distributed hain: $= [E [e^{t Y_{1} / n}]]^{n} = [M_{Y_{1}} (\frac{t}{n})]^{n}$

Step 4: Taylor Expansion

Taylor series kyun? Jab $n \to \infty$ , $t / n \to 0$ , toh hum MGF ko approximate kar sakte hain.

Chhote $s$ ke liye: $M_{Y_{1}} (s) = E [e^{s Y_{1}}] = E [1 + s Y_{1} + \frac{s ^{2} Y _{1}^{2}}{2} + \frac{s ^{3} Y _{1}^{3}}{6} + \dots]$

$= 1 + s E [Y_{1}] + \frac{s ^{2}}{2} E [Y_{1}^{2}] + \frac{s ^{3}}{6} E [Y_{1}^{3}] + O (s^{4})$

Kyunki $E [Y_{1}] = 0$ aur $E [Y_{1}^{2}] = 1$ : $M_{Y_{1}} (s) = 1 + \frac{s ^{2}}{2} + O (s^{3})$

$s = t / n$ substitute karo: $M_{Y_{1}} (\frac{t}{n}) = 1 + \frac{t ^{2}}{2 n} + O (\frac{1}{n ^{3/2}})$

Step 5: Limit Lena

$M_{Z_{n}} (t) = [1 + \frac{t ^{2}}{2 n} + O (\frac{1}{n ^{3/2}})]^{n}$

Yeh $e^{t^{2} /2}$ kyun approach karta hai? $e^{x}$ ki limit definition yaad karo: $lim_{n \to \infty} (1 + \frac{x}{n})^{n} = e^{x}$

Yahan $x = t^{2} /2$ : $lim_{n \to \infty} M_{Z_{n}} (t) = lim_{n \to \infty} (1 + \frac{t ^{2} /2}{n})^{n} = e^{t^{2} /2}$

Yeh $N (0, 1)$ ka MGF hai. Therefore $Z_{n} d N (0, 1)$ . ∎

Examples

Worked example Example 1: Uniform Die Rolls

Setup: Ek fair die $n = 36$ baar roll karo. Probability nikalo ki average 3.3 aur 3.7 ke beech ho. YEH kyun matter karta hai?: Non-normal (uniform) source ke saath CLT demonstrate karta hai.

Step 1: Parameters identify karo

Single die: $X \sim Uniform {1, 2, 3, 4, 5, 6}$
$μ = \frac{1 + 2 + 3 + 4 + 5 + 6}{6} = 3.5$
$σ^{2} = E [X^{2}] - μ^{2} = \frac{1 + 4 + 9 + 16 + 25 + 36}{6} - 3. 5^{2} = \frac{91}{6} - 12.25 = 2.9167$
$σ = 1.708$

YEH variance formula kyun? $Var (X) = E [X^{2}] - (E [X])^{2}$ by definition.

Step 2: CLT apply karo Sample mean $\overset{ˉ}{X}_{36} \sim N (3.5, \frac{2.9167}{36}) = N (3.5, 0.081)$

Standard error: $SE = \frac{1.708}{36} = 0.285$

Step 3: Standardize karo $P (3.3 \leq \overset{ˉ}{X}_{36} \leq 3.7) = P (\frac{3.3 - 3.5}{0.285} \leq Z \leq \frac{3.7 - 3.5}{0.285})$

$μ$ kyun subtract karte hain aur $SE$ se kyun divide karte hain? Standard normal mein convert karne ke liye.

$= P (- 0.70 \leq Z \leq 0.70) = Φ (0.70) - Φ (- 0.70)$ $= 0.758 - 0.242 = 0.516$

Answer: Lagbhag 52% probability.

Worked example Example 2: ML Gradient Estimation

Setup: Ek neural network train kar rahe ho, $n = 64$ samples ke mini-batch se gradient estimate karte ho. Har sample ke gradient ka mean $μ_{g} = - 0.03$ aur variance $σ_{g}^{2} = 0.25$ hai. Kya probability hai ki tumhara gradient estimate true gradient ke ±0.01 ke andar ho?

YEH kyun matter karta hai?: Explain karta hai kyun stochastic gradient descent noisy gradients ke bawajood converge karta hai.

Step 1: Sample mean gradient $\overset{g}{ˉ}_{64} \sim N (- 0.03, \frac{0.25}{64}) = N (- 0.03, 0.0039)$

$SE = \frac{0.25}{8} = \frac{0.5}{8} = 0.0625$

Step 2: Probability compute karo $P (- 0.04 \leq \overset{g}{ˉ}_{64} \leq - 0.02) = P (\frac{- 0.04 - ( - 0.03 )}{0.0625} \leq Z \leq \frac{- 0.02 - ( - 0.03 )}{0.0625})$ $= P (- 0.16 \leq Z \leq 0.16) \approx 0.127$

Itna kam kyun? ±0.01 ki window standard error ke relative narrow hai. 95% accuracy ±0.01 ke andar paane ke liye tumhe chahiye: $1.96 \times \frac{0.5}{n} \leq 0.01 ⟹ n \geq (\frac{1.96 \times 0.5}{0.01})^{2} = 9604$

Insight: Bade mini-batches zyada accurate gradient estimates dete hain lekin zyada computation cost karte hain.

Worked example Example 3: A/B Test Sample Size

Setup: Tum ek naya recommender algorithm test kar rahe ho. Conversion rate 5% hai (Bernoulli with $p = 0.05$ ). 95% confidence ke saath true rate ke 0.5% ke andar sample mean paane ke liye kitne users chahiye? YEH kyun matter karta hai?: Experiments ke liye Sample size calculation.

Step 1: Bernoulli parameters

$μ = p = 0.05$
$σ^{2} = p (1 - p) = 0.05 \times 0.95 = 0.0475$ Step 2: Confidence interval 95% confidence ke liye: $P (∣ \overset{ˉ}{X}_{n} - μ ∣ \leq 0.005) = 0.95$

0.005 kyun? Yeh 0.5% proportion ke roop mein express kiya gaya hai.

Standardize karo: $P (\frac{X ˉ _{n} - μ}{σ / n} \leq \frac{0.005}{σ / n}) = 0.95$

Standard normal ke liye: $P (∣ Z ∣ \leq 1.96) = 0.95$

1.96 kyun? 95% two-tailed test ke liye critical value.

Step 3: $n$ ke liye solve karo $\frac{0.005}{σ / n} = 1.96$ $n = \frac{1.96 \times 0.0475}{0.005} = \frac{1.96 \times 0.218}{0.005} = 85.4$ $n = 7293$

Answer: Har variant ke liye ~7300 users chahiye.

Common Mistakes

Common mistake Mistake 1: "CLT ka matlab data normal ho jaata hai"

Galat reasoning: "Agar main enough data collect karun, toh underlying distribution normal ho jaati hai."

YEH sahi kyun lagta hai?: Hum large samples ke saath normal curves appear hote dekhte hain.

Sach yeh hai: Sample mean ki distribution normal hoti hai, lekin individual data points apni original distribution rakhte hain. 1000 dice roll karne se ek single die roll normal nahi hota—woh phir bhi uniform {1,2,3,4,5,6} rehta hai.

Steel-man: Confusion isliye hoti hai kyunki hum aksar data ke histograms plot karte hain (jo source ke true rehte hain) vs. means jaisi statistics ki distributions (jo normal ho jaati hain). ML mein, features sirf isliye normal nahi ho jaate kyunki tumhare paas zyada samples hain—lekin unke sample means ho jaate hain.

Fix: CLT aggregations (means, sums) par apply hota hai, individual observations par nahi.

Active Recall Questions

Recall Feynman Explanation: 12-saal ke bachche ko padhaao

Socho tumhare paas alag-alag colored marbles ki ek bucket hai—kuch red, kuch blue, kuch green, sab randomly mixed. Agar tum sirf EK marble pakdo, tumhe pata nahi kya milega. Kuch bhi ho sakta hai!

Lekin yahan magic hai: Agar tum 30 marbles pakdo aur count karo kitne reds mile, phir unhe wapas rakh do aur yeh baar baar karo, kuch ajeeb hota hai. Chahe bucket random chaos hai, har baar tumhare ginne wale reds ki sankhya ek pattern follow karne lagti hai—ek bell curve! Zyada tar time tum average ke paas pahunchoge, rarely bahut zyada ya bahut kam milenge.

Yeh kyun matter karta hai? Iska matlab hai ki jab individual cheezein bilkul random aur unpredictable hoti hain, jab tum bahut saari cheezein ke averages dekhte ho, woh predictable ho jaate hain! Isliye scientists survey results par trust kar sakte hain chahe individual log alag hain. Isliye tumhari teacher test fairly grade kar sakti hai—tumhari class average unhe kuch real batata hai, chahe har student unique hai.

Central Limit Theorem nature ka yeh kehne ka tarika hai: "Chaos + Averaging = Order."

#flashcards/ai-ml

Central Limit Theorem ek sentence mein kya hai? :: CLT kehta hai ki sample means ki distribution normal distribution ki taraf approach karti hai jab sample size badhta hai, chahe population ki distribution koi bhi ho (finite variance assuming).

CLT apply hone ke liye teen requirements kya hain?

1) Samples independent hone chahiye, 2) Samples identically distributed hone chahiye (same population), 3) Population variance finite honi chahiye.

Sample mean ka standard error sample size ke saath kaise change hota hai? :: Standard error = σ/√n, toh yeh n ke square root ke proportionally decrease hota hai. Error aadha karne ke liye 4× sample size chahiye.

Agar population std dev σ = 10 hai aur n = 100, toh standard error kya hai?

SE = 10/√100 = 10/10 = 1

Machine learning ke liye CLT kyun matter karta hai?

CLT justify karta hai gradient descent use karna (noisy gradient estimates converge karte hain), model performance ke liye confidence intervals, hypothesis testing, aur non-normal data par bhi predictions par statistical inference.

Standardized sample mean kaun si distribution follow karta hai?

Standard normal distribution N(0, 1) jab n → ∞.

Sample mean standardize karne ka formula kya hai?

Z = (X̄ₙ - μ)/(σ/√n)

Kya CLT underlying data ko normal banata hai?

Nahi! CLT sample means ki distribution ko normal banata hai. Individual data points apni original distribution rakhte hain.

"n ≥ 30" rule kya hai aur yeh kab fail hota hai?

Common heuristic ki n ≥ 30 ke liye CLT approximation achhi hai. Heavily skewed ya heavy-tailed distributions ke liye fail hoti hai (bahut bada n chahiye) aur nearly-normal distributions ke liye (chhote n ke liye kaam karta hai).

95% confidence interval ke liye, sample mean μ ke kitne standard errors ke andar hota hai?

±1.96 standard errors ke andar (standard normal se critical value).

Connections

Law of Large Numbers: Related lekin distinct—LN kehta hai sample mean population mean ki taraf converge karta hai; CLT us convergence ki distribution describe karta hai
Normal Distribution: CLT explain karta hai kyun Gaussians nature mein har jagah appear hote hain—woh averaged processes ki limit hain
Confidence Intervals: CLT sample means ke aas-paas CIs construct karne ke liye theoretical foundation provide karta hai
Hypothesis Testing: t-tests aur z-tests large samples ke saath validity ke liye CLT par rely karte hain
Stochastic Gradient Descent: CLT explain karta hai kyun noisy mini-batch gradients convergence lead karte hain
Bootstrap Methods: Non-parametric alternative jab CLT assumptions (independence, finite variance) fail ho jaayein
Sample Size Calculation: Desired precision ke liye kitna data chahiye yeh determine karne ke liye CLT use karta hai
Monte Carlo Methods: Simulation accuracy CLT se improve hoti hai—bahut saare random samples average karna
Maximum Likelihood Estimation: MLEs ki asymptotic normality CLT ka consequence hai
Bias-Variance Tradeoff: CLT explain karta hai kyun estimators ka variance O(1/n) se decrease hota hai