Exercises — Law of large numbers

(b). Weak Law ek probability ke zero tak shrink hone ki baat karta hai jaise jaise $n$ badhta hai: kisi bhi tolerance $ϵ$ ke liye, $\mu$ se itna miss hone ka chance fade hota jaata hai. Option (a) Strong Law hai (ek infinite run ke hamesha ke liye settle hone ka statement). Option (c) bilkul galat hai — average kabhi exactly $\mu$ nahi hota, wo sirf uske paas concentrate hota hai. Answer ::: (b)

$\mathrm{Var}({\bar{X}}_n)=\frac{{\sigma }^2}{n}=\frac{9}{100}=0.09$. Standard error uska square root hai: $\mathrm{SE}=\sqrt{0.09}=0.3$. Equivalently $\sigma /\sqrt{n}=3/10=0.3$. Average ek single draw se $10\times$ zyada tight hai ($\sigma =3$). Var ::: 0.09 SE ::: 0.3

False. $E[{\bar{X}}_n]=\mu$ har $n$ ke liye — averaging kabhi centre shift nahi karta, sirf spread shrink karta hai. Yahi reason hai ki average $\mu$ ka ek unbiased estimate hai (dekho Expected Value aur Bias-Variance Tradeoff). Answer ::: False

$\frac{{\sigma }^2}{n{ϵ}^2}$ mein ${\sigma }^2=35/12$, $n=600$, $ϵ=0.2$ plug karo: $P(|{\bar{X}}_n-3.5|\ge 0.2)\le \frac{35/12}{600\cdot (0.2{)}^2}=\frac{2.91667}{600\cdot 0.04}=\frac{2.91667}{24}\approx \mathrm{0.1215.}$ To average roll ka $0.2$ ya zyada off hone ka chance zyada se zyada 12.2% hai. Bound ::: ≈ 0.1215

Chebyshev bound ko target ke barabar set karo: $\frac{0.25}{n(0.05{)}^2}\le 0.05$. $(0.05{)}^2=0.0025$ compute karo, to left side hai $\frac{0.25}{0.0025n}$. Ab $\frac{0.25}{0.0025}=100$ (kyunki $0.25=100\times 0.0025$), aur $\frac{100}{n}\le 0.05$ milta hai. Solve karo: $n\ge \frac{100}{0.05}=2000$. $n\ge 2000$ flips chahiye (Chebyshev conservative hai; tighter Central Limit Theorem ke liye bahut kam chahiye). n ::: 2000

Chebyshev: $\frac{{\sigma }^2}{n{ϵ}^2}\le 0.10$ jisme ${\sigma }^2=16$, $ϵ=0.5$, ${ϵ}^2=0.25$. Left side hai $\frac{16}{0.25n}$. Kyunki $\frac{16}{0.25}=64$ (kyunki $0.25$ se divide karna $4$ se multiply karne jaisa hai, aur $16\times 4=64$), yeh $\frac{64}{n}\le 0.10$ ban jaata hai. $\frac{64}{n}\le 0.10\Rightarrow n\ge \frac{64}{0.10}=640.$ Minimum $n=640$ readings. n ::: 640

$\mathrm{SE}=\sigma /\sqrt{n}$. $\mathrm{SE}$ ko half karne ke liye $\sqrt{n_2}=2\sqrt{n_1}$ chahiye, yani $n_2=4n_1$. To $n$ ko 4 se multiply karo ($1000$ se $4000$).

Neeche wali figure standard error $\mathrm{SE}=\sigma /\sqrt{n}$ ko sample size $n$ ke against plot karti hai ($\sigma =1$ ke saath). Magenta curve follow karo: $n$ double karne se height mein zyada fark nahi padta, aur do marked points dikhate hain ki error half karne ke liye $n=1000$ se seedha $n=4000$ (violet aur orange dots) tak jaana padta hai. Curve right side pe brutally flat ho jaata hai — woh flatness hi "precision quadratically costly hai" ka lesson hai, kyunki accuracy ka aakhri slice sabse zyada data maangta hai.

${\mathrm{SE}}_A=\frac{2}{\sqrt{100}}=\frac{2}{10}=0.2$. ${\mathrm{SE}}_B=\frac{5}{\sqrt{400}}=\frac{5}{20}=0.25$. A zyada tight hai ($0.2<0.25$), bhalee B ke paas $4\times$ zyada samples hain — kyunki B ka per-draw noise $\sigma$ $2.5\times$ bada hai. Sample size aur variance dono matter karte hain; zyada data hamesha noisier source ko nahi outrun kar sakta. Smaller SE ::: Estimator A (0.2 vs 0.25)

Pehle loose bound reproduce karo taaki yeh self-contained rahe: ${\sigma }^2=0.25$, $n=10,000$, $ϵ=0.01$ ke saath, $P(|{\bar{X}}_n-0.5|\ge 0.01)\le \frac{{\sigma }^2}{n{ϵ}^2}=\frac{0.25}{10000\cdot (0.01{)}^2}=\frac{0.25}{10000\cdot 0.0001}=\frac{0.25}{1}=\mathrm{0.25.}$ Chebyshev sirf variance use karta hai — yeh distribution ki shape ke baare mein koi assumption nahi karta, to isse worst-case shape ke liye safe rehna padta hai. Woh safety sharpness ki cost par aati hai, isliye loose $0.25$ milta hai.

Central Limit Theorem ek real assumption add karta hai: large $n$ ke liye, ${\bar{X}}_n$ approximately normal hai jisme SE $=\sigma /\sqrt{n}=0.5/\sqrt{10000}=0.005$. Tab $ϵ=0.01$ $2$ standard errors door hai, aur $2$ SE se pare two-tailed normal tail $\approx 0.0455$ hai. Kyunki CLT bell shape jaanta hai, woh bahut chhota number de sakta hai. Tighter theorem ::: Central Limit Theorem (normal shape use karta hai, sirf variance nahi)

Worst-case variance ${\sigma }^2=0.25$ use karo taaki guarantee chahe koi bhi $g$ ho hold kare. $\frac{0.25}{n(0.002{)}^2}\le \mathrm{0.05.}$ $(0.002{)}^2=4\times 10^{-6}$, to left side hai $\frac{0.25}{4\times 10^{-6}n}$. Ab $\frac{0.25}{4\times 10^{-6}}=62500$ (kyunki $0.25÷0.000004=62500$), to $\frac{62500}{n}\le 0.05$ milta hai. $n\ge \frac{62500}{0.05}=1,250,000$. 1,250,000 samples chahiye. Yeh Monte Carlo Methods ke peeche ka pessimistic Chebyshev count hai; CLT-based error bars practice mein bahut chhote hote hain. n ::: 1,250,000

Averaged-gradient noise ${\sigma }_g/\sqrt{b}$ ki tarah scale hoti hai, to noise $\propto 1/\sqrt{b}$. Noise ko $4$ se divide karne ke liye $\sqrt{b_2}=4\sqrt{b_1}$ chahiye, yani $b_2=16b_1=16\times 16=256$. Batch size 256. Gaur karo ki $16\times$ zyada compute sirf $4\times$ kam noise deta hai — wahi $1/\sqrt{n}$ tax jo saare LLN estimates par lagta hai, aur isiliye bahut bade batches diminishing returns dikhate hain. Batch size ::: 256

Bootstrap ka khud ka Monte-Carlo error bhi $\propto 1/\sqrt{B}$ follow karta hai. $0.02$ ko $0.005$ tak kaat na $4$ ka factor hai, jiske liye $\sqrt{B_2}=4\sqrt{B_1}$ chahiye, to $B_2=16B_1=16\times 100=1600$. Lagbhag 1600 resamples. B ::: 1600

Nahi — yeh converge nahi karta. Jo bhi Law humne build kiya usme finite $\mu$ assume kiya gaya hai (aur Chebyshev ko bhi finite ${\sigma }^2$ chahiye). Cauchy distribution pehli hi hypothesis violate karta hai, isliye Law apply nahi hota, aur actually ${\bar{X}}_n$ hamesha ke liye wandering karta rehta hai — ek single extreme draw chahe $n$ kitna bhi bada ho, pure average ko dominate kar sakta hai. Lesson: guarantee conditional hai. Heavy tails (infinite variance, ya worse, infinite mean) LLN ko break karte hain. Average par trust karne se pehle hamesha hypotheses check karo. Converges? ::: Nahi — mean undefined hai, LLN ki assumptions fail hoti hain

Maano $S_n=\sum X_i$ = total heads, ${\bar{X}}_n=S_n/n$ = fraction. Yaad raho SD = standard deviation = variance ka square root.

• Fraction: $\mathrm{SE}({\bar{X}}_n)=\sigma /\sqrt{n}=0.5/\sqrt{n}$. Yeh $1/\sqrt{n}$ ki tarah shrink karta hai — $n$ ko $4$ se multiply karo aur yeh half ho jaata hai — to fraction $0.5$ par concentrate karta hai. ✓
• Count: $\mathrm{SD}(S_n)=\sigma \sqrt{n}=0.5\sqrt{n}$. Yeh $\sqrt{n}$ ki tarah grow karta hai — $n$ ko $4$ se multiply karo aur yeh double ho jaata hai — to yeh infinity ki taraf jaata hai.

To heads-count aur $n/2$ ke beech typical gap $\sqrt{n}$ ki tarah grow karta hai, jabki fraction ka $0.5$ se gap $1/\sqrt{n}$ ki tarah shrink karta hai. Dono ek saath sach hain — yahi reason hai ki "mujhe exactly $n/2$ heads milenge" galat hai. Fraction error growth ::: 1/√n ki tarah shrink karta hai Count error growth ::: √n ki tarah grow karta hai

Fair coin ke under, ${\bar{X}}_n$ ka $\mu =0.5$ aur $\mathrm{SE}=0.5/\sqrt{100}=0.05$ hai. Observed fraction $0.6$ yahan baith ta hai: $\frac{0.6-0.5}{0.05}=\frac{0.1}{0.05}=2\text{standard errors above the mean.}$ $2$ SE ya zyada ka deviation fair coin ke liye roughly $5\%$ time hota hai (normal approximation). Yeh bias ka strong evidence nahi hai — yeh mildly unusual lekin bilkul ordinary result hai. "Biased!" conclude karna yahan gambler's-fallacy reflex hai; aapko ek proper confidence interval banana chahiye aur coin par doubt karne se pehle bahut bada deviation (say $3$ SE ya zyada) maangna chahiye. Conclusion: $z=2$ → strong evidence nahi; fair coin aisa result roughly $5\%$ time deta hai. z ::: 2 standard errors → strong evidence nahi

• Kya aap WLLN vs SLLN words mein restate kar sakte ho? ::: WLLN: $\mu$ ko $ϵ$ se miss karne ki probability → 0. SLLN: ek single run probability 1 ke saath $\mu$ par settle hoti hai.
• Error half karne ke liye $n$ ko kitne se multiply karna hoga? ::: 4.
• Law ko poori tarah kya tod deta hai? ::: Infinite mean ya infinite variance (heavy tails, e.g. Cauchy).