Foundations — Law of large numbers

1.3.14 · D1 · AI-ML › Probability & Statistics › Law of large numbers

Pehle hum parent note ke formulas ko padh bhi nahin sakte jab tak hum notation ka har piece earn na kar lein. Hum order mein chalenge: ek single random draw se, draws ki ek list tak, unke average tak, aur phir un do Greek letters tak jo describe karte hain ki woh average kahan ja raha hai aur kitna wildly wobble karta hai. Har naye symbol ko ek plain meaning, ek picture, aur ek reason milega ki topic uske bina zinda nahin reh sakta.

---

1. Ek random variable — symbol $X$

Simple words mein: ek coin flip karo — outcome hai "heads" ya "tails," jo words hain, numbers nahin. Ek random variable woh translator hai jo kehta hai "heads $\to 1$, tails $\to 0$." Ab har flip ek number produce karta hai jise hum average kar sakte hain.

Picture: ek machine socho jisme ek slot hai. Tum ek random event (ek coin toss) andar daalo, aur neeche se ek number nikalta hai.

Topic ko yeh kyun chahiye: Law of Large Numbers poori tarah numbers average karne ke baare mein hai. Tum "heads" word ka average nahin nikal sakte. $X$ woh bridge hai "stuff happening" se "arithmetic we can do" tak.

Recall

Yahan capital $X$ kis cheez ke liye khada hai? ::: Ek random variable ke liye — ek rule jo har random outcome ko ek number assign karta hai.

---

2. Bahut saare draws — subscripts $X_1,X_2,\dots ,X_n$

Simple words mein: subscript sirf ek naam tag hai. $X_5$ "$X$ times 5" nahin hai — yeh hai "paanchwa draw." $i$ ek placeholder hai jo $1$ se $n$ tak koi bhi tag ho sakta hai.

Picture: buckets ki ek row, har ek mein ek draw ka number, left se right label kiye gaye.

Topic ko yeh kyun chahiye: poori kahaani hai "kya hota hai jab $n$ barhta hai." Hume draw number $1$, draw number $1,000,000$, aur beech ke har cheez ke baare mein baat karne ka tarika chahiye — subscripts hume woh vocabulary dete hain.

---

3. Summation sign — ${\sum }_{i=1}^n{X}_i$

Simple words mein: yeh ek lambi "$+++$" chain likhne ka compact tarika hai bina exactly jaane ki kitne terms hain.

Yeh tool kyun aur sirf "$+$" kyun nahin? Kyunki $n$ bahut bada ya unknown ho sakta hai. Hum literally ek million plus-signs nahin likh sakte, aur hum chahte hain ek aisa formula jo kisi bhi $n$ ke liye kaam kare. Sigma woh tool hai jo "inhe saare jodo" ko ek clean symbol mein kehta hai.

Recall

${\sum }_{i=1}^n$ mein, neeche ka "1" aur upar ka "$n$" kya batate hain? ::: Counting kahan se shuru karein (draw 1) aur kahan rokein (draw $n$).

---

4. Sample mean — ${\bar{X}}_n$

Simple words mein: yeh wahi ordinary "sab jodo aur divide karo" average hai jo tum already jaante ho, sirf hamare naye symbols se likha gaya hai. Total heads divided by total flips = heads ka proportion.

Picture: saare bucket-numbers ko ek seesaw par rakh do. Us seesaw ka balance point hai ${\bar{X}}_n$.

Topic ko yeh kyun chahiye: ${\bar{X}}_n$ poore show ka star hai. Law of Large Numbers ek statement hai iske baare mein ki jab $n$ barhta hai toh yeh quantity kya karti hai. Baki sab kuch describe karne ke liye exist karta hai ki ${\bar{X}}_n$ kahan jaata hai.

---

5. Expected value — symbol $\mu =E[X]$

Simple words mein: ek fair coin ke liye, aadhe flips heads ($1$) hain aur aadhe tails ($0$), toh forever-average hai $0.5$. Woh $0.5$ hai $\mu$. ${\bar{X}}_n$ se farq note karo: sample mean ${\bar{X}}_n$ woh hai jo tumne $n$ real flips se actually measure kiya (yeh wobble karta hai); $\mu$ woh ideal target hai jiske aas-paas yeh wobble karta hai (yeh kabhi nahin hilta).

Picture: number line par, $\mu$ ek fixed flag hai jo zameen mein gada hai. ${\bar{X}}_n$ ek dot hai jo flag ke paas naachta hai aur, jab $n$ barhta hai, aur paas naachta hai.

Topic ko yeh kyun chahiye: Law of Large Numbers precisely "${\bar{X}}_n$ $\mu$ ki taraf jaata hai" hai. Bina target ke symbol ke, hamare paas kuch nahin hai jis taraf converge karein. $E[X]$ ki poori construction ke liye Expected Value dekho.

Recall

Jab tum data collect karte ho toh kaunsa move karta hai, $\mu$ ya ${\bar{X}}_n$? ::: ${\bar{X}}_n$ move karta hai (yeh measured hai); $\mu$ ek fixed target hai.

---

6. Variance aur ${\sigma }^2$ — wobble kitna wide hai

Simple words mein: do experiments ek hi target $\mu$ share kar sakte hain phir bhi bilkul alag feel kar sakte hain — ek shaant, ek chaotic. ${\sigma }^2$ woh number hai jo us chaos ko capture karta hai. Yeh mean se squared distances ka average hai (squaring har distance ko positive banata hai aur bade misses ko zyada punish karta hai).

Topic ko yeh kyun chahiye: parent ka key formula hai $\text{Var}({\bar{X}}_n)={\sigma }^2/n$. Woh single fact — average ka wobble shrink karta hai jab $n$ barhta hai — poore proof ka engine hai. Tum "zyada data kyun help karta hai" nahin samajh sakte bina ${\sigma }^2$ ke. Full treatment Variance and Standard Deviation mein.

Recall

Chhota ${\sigma }^2$ draws ke baare mein kya batata hai? ::: Woh mean $\mu$ ke aas-paas tightly cluster karte hain.

---

7. Kisi event ki probability — $P(\cdot )$

Parent aisi cheezein likhta hai jaise $P(|{\bar{X}}_n-\mu |\ge ϵ)$. Ise inside-out padho:

• ${\bar{X}}_n-\mu$ = tumhara measured average true target se kitna door hai.
• $|\cdots |$ = absolute value: sign hatao, sirf distance rakho (miss $-0.3$ ho ya $+0.3$, dono equally "off" hain).
• $\ge ϵ$ = "woh distance kam se kam $ϵ$ hai" (ek chhota threshold — Greek letter epsilon, matlab "ek tiny amount jo tum choose karo").
• $P(\dots )$ = woh chance ki yeh sab sach ho.

Toh poora expression matlab hai: "woh probability ki sample mean target ko ek whisker $ϵ$ se zyada miss kare." Law kehta hai ki yeh probability $0$ par crash kar jaati hai jab $n$ barhta hai.

Recall

$|{\bar{X}}_n-\mu |$ mein absolute-value bars kyun hain? ::: Hum target se distance ki parwah karte hain, direction ki nahin — upar aur neeche dono "off" count hote hain.

---

8. Limit — ${\lim }_{n\to \infty }$

Simple words mein: fraction ${\sigma }^2/(n{ϵ}^2)$ har baar chhota hota jaata hai jab $n$ barhta hai. $n=10$ daalo, phir $100$, phir $1,000,000$: value $0$ ki taraf march karti hai lekin kabhi ek fixed stopping point nahin hoti — woh bas wahan jaati hai. Limit us march ki destination hai.

Topic ko yeh kyun chahiye: "zyada aur zyada repetitions" hi hai $n\to \infty$. Law of Large Numbers ek statement hai limit ke baare mein, isliye yeh symbol unavoidable hai.

---

9. i.i.d. — draws par fine print

Topic ko yeh kyun chahiye: independence precisely woh cheez hai jo Step 3 mein parent ke proof mein variances ko add karne deti hai ($\text{Var}$ of a sum = sum of $\text{Var}$s). "Identical" woh hai jo hume ek shared $\mu$ aur ${\sigma }^2$ likhne deta hai saare draws ke liye. Koi bhi todlo aur proof collapse ho jaata hai. Deep dive: Independent and Identically Distributed.

---

Yeh theorem ko kaise feed karte hain

Upar se neeche padho: numbers ($X$) ek list ban jaate hain, list ek average ban jaati hai (${\bar{X}}_n$), hum target se uski distance measure karte hain ($\mu$), variance batata hai ki wobble shrink karta hai, aur limit "shrinks" ko "vanishes" mein badal deti hai — jo exactly Law of Large Numbers hai.

---

Yeh tools kahan dobara dikhte hain

Jab yeh symbols tumhare paas ho jaate hain, toh yeh neighbours unlock ho jaate hain: Chebyshev Inequality ${\sigma }^2$, $ϵ$, aur $P(\cdot )$ use karta hai miss-probability ko bound karne ke liye; Central Limit Theorem "${\bar{X}}_n$ near $\mu$" ko ek exact bell shape mein refine karta hai; Monte Carlo Methods aur Bootstrap Sampling averaging idea ko estimation algorithms mein turn karte hain; aur Stochastic Gradient Descent usi "mini-batch average $\to$ true average" logic par lean karta hai.

---

Equipment checklist

Apne aap ko test karo — right side cover karo aur reveal karne se pehle jawab do.

• Capital $X$ ka matlab hai... ::: ek random variable: ek rule jo ek random outcome ko ek number mein badalta hai.
• $X_i$ mein subscript ka matlab hai... ::: kaunsa draw (ek naam tag), multiplication nahin.
• ${\sum }_{i=1}^n{X}_i$ expand hota hai... ::: $X_1+X_2+\cdots +X_n$ mein, saare $n$ draws ka sum.
• ${\bar{X}}_n$ compute kiya jaata hai... ::: saare $n$ draws jodkar aur $n$ se divide karke — sample average.
• $\mu =E[X]$ hai... ::: woh fixed long-run (forever) average, woh target jise ${\bar{X}}_n$ chase karta hai.
• $\mu$ aur ${\bar{X}}_n$ mein farq hai... ::: $\mu$ fixed ideal hai; ${\bar{X}}_n$ wobbling measured value hai.
• ${\sigma }^2$ measure karta hai... ::: variance — draws $\mu$ ke aas-paas kitni widely spread hain.
• $|{\bar{X}}_n-\mu |\ge ϵ$ describe karta hai... ::: average ka target ko kam se kam ek chhote threshold $ϵ$ se miss karna.
• $P(\text{event})$ ek number hai range mein... ::: $0$ (impossible) se $1$ (certain) tak.
• ${\lim }_{n\to \infty }$ poochta hai... ::: expression kis value ki taraf approach karta hai jab $n$ bina bound ke barhta hai.
• "i.i.d." require karta hai... ::: draws jo independent hain aur ek hi distribution share karte hain ($\mu$, ${\sigma }^2$).