Chi-squared, t, F distributions — definition, degrees of freedom

4.9.9 · Maths › Probability Theory & Statistics

---

1. The Chi-squared distribution ${\chi }_k^2$

Mean aur variance derive karna (scratch se)

YE KYU KAAM KARTA HAI: expectation linear hai, aur hume sirf ek $Z^2$ ke baare mein facts chahiye.

Ek $Z\sim N(0,1)$ ke liye: $E[Z^2]=\mathrm{Var}(Z)+(E[Z]{)}^2=1+0=1.$ Ye step kyun? Kyunki $\mathrm{Var}(Z)=E[Z^2]-(E[Z]{)}^2$, rearrange karne par.

Ek standard normal ka fourth moment $E[Z^4]=3$ hai, isliye $\mathrm{Var}(Z^2)=E[Z^4]-(E[Z^2]{)}^2=3-1=2.$

Ab $k$ independent copies sum karo:

\operatorname{Var}(\chi^2_k) = \sum_{i=1}^k \operatorname{Var}(Z_i^2) = 2k.$$ *Ye step kyun?* Variance independent terms pe add hota hai; wahi independence df count ka poora point hai. > [!formula] Chi-squared facts > $$\chi^2_k = \sum_{i=1}^k Z_i^2,\qquad E[\chi^2_k]=k,\qquad \operatorname{Var}(\chi^2_k)=2k.$$ > **Additive:** $\chi^2_a + \chi^2_b = \chi^2_{a+b}$ agar independent hain (bas aur squared normals append karo). > **Key statistics result:** size $n$ ke sample ke liye $N(\mu,\sigma^2)$ se, > $$\frac{(n-1)s^2}{\sigma^2}\sim \chi^2_{n-1},\quad s^2=\tfrac{1}{n-1}\sum(x_i-\bar x)^2.$$ > Df $n$ nahi $n-1$ hai, kyunki $\bar x$ estimate karne se $\sum(x_i-\bar x)=0$ impose hoti hai — ==ek constraint, ek df lose==. --- ## 2. The Student's t distribution $t_k$ > [!definition] Definition > Agar $Z\sim N(0,1)$ aur $V\sim\chi^2_k$ **independent** hain, tab > $$ t_k = \frac{Z}{\sqrt{V/k}} $$ > $k$ degrees of freedom ke saath $t$ distribution follow karta hai. Denominator ek $\chi^2$ hai ==apne df se divided, phir square-rooted== — yaani ek estimated standard deviation. > [!intuition] ISKA normal se fatter tails kyun hain > $t = Z/\sqrt{V/k}$ mein, denominator **random** hai (yeh $\sigma$ ka estimate hai, true $\sigma$ nahi). Kabhi kabhi denominator chance se chota hota hai → $t$ blow up hota hai → zyada extreme values → **heavier tails**. Jab $k\to\infty$, $V/k\to 1$ (law of large numbers) to denominator 1 pe stabilise ho jaata hai aur $t_k \to Z$. ### Practice mein yeh kahan se aata hai (derivation sketch) Hum ek mean test karna chahte hain **unknown variance** ke saath. Standardised sample mean se shuru karo: $$\frac{\bar x - \mu}{\sigma/\sqrt n}\sim N(0,1)=Z.$$ *Kyun?* $\bar x\sim N(\mu,\sigma^2/n)$, phir standardise karo. Lekin $\sigma$ unknown hai — use $s$ se replace karo: $$\frac{\bar x-\mu}{s/\sqrt n} = \frac{(\bar x-\mu)/(\sigma/\sqrt n)}{s/\sigma} = \frac{Z}{\sqrt{\dfrac{(n-1)s^2/\sigma^2}{n-1}}} = \frac{Z}{\sqrt{V/k}},\quad V=\tfrac{(n-1)s^2}{\sigma^2}\sim\chi^2_{n-1}.$$ *Ye step kyun?* Hum numerator aur denominator dono ko $\sigma$ se divide karte hain taaki upar exactly $Z$ bane aur neeche ek $\chi^2_{n-1}$ apne df se scaled ho — definition se match karta hai $k=n-1$ ke saath. > [!formula] t-distribution facts > $$t_k=\frac{Z}{\sqrt{V/k}},\quad E[t_k]=0\ (k>1),\quad \operatorname{Var}(t_k)=\frac{k}{k-2}\ (k>2).$$ > 0 ke around symmetric, bell-shaped, $N(0,1)$ se fatter tails; $t_k\to N(0,1)$ jab $k\to\infty$. --- ## 3. The F distribution $F_{d_1,d_2}$ > [!definition] Definition > Agar $U\sim\chi^2_{d_1}$ aur $V\sim\chi^2_{d_2}$ **independent** hain, tab > $$ F_{d_1,d_2} = \frac{U/d_1}{V/d_2} $$ > **numerator df $d_1$** aur **denominator df $d_2$** ke saath $F$ distribution hai. Yeh ==do chi-squareds ka ratio hai, har ek apne df se divided==. > [!intuition] HAR EK ko apne df se divide KYUN karte hain > Ek raw $\chi^2_k$ $k$ ke saath badhta hai (mean $=k$). Dono ko *fairly* compare karne ke liye aap har ek ko "per degree of freedom" normalise karte ho, taaki har part ka mean $\approx 1$ ho. Tab $F$ measure karta hai ki ek normalised variance estimate **kitni baar badi** hai doosri se. Agar do populations ki variance equal hai, to $F\approx 1$. ### Teeno ke beech connections (derive karo, memorise mat karo) **$t^2$ ek $F$ hai:** $$t_k^2 = \frac{Z^2}{V/k} = \frac{Z^2/1}{V/k} = F_{1,k}.$$ *Kyun?* $Z^2\sim\chi^2_1$, isliye squared $t$ ek $\chi^2_1$ (df=1) hai ek $\chi^2_k/k$ ke upar — exactly $F_{1,k}$. **Reciprocal symmetry:** numerator/denominator swap karne se $F$ invert ho jaata hai: $$\frac{1}{F_{d_1,d_2}} = \frac{V/d_2}{U/d_1}\sim F_{d_2,d_1}.$$ > [!formula] F-distribution facts > $$F_{d_1,d_2}=\frac{U/d_1}{V/d_2},\qquad E[F_{d_1,d_2}]=\frac{d_2}{d_2-2}\ (d_2>2).$$ > ANOVA mein aur do sample variances $s_1^2/s_2^2$ compare karne ke liye use hota hai. ![[4.9.09-Chi-squared,-t,-F-distributions-—-definition,-degrees-of-freedom.png]] --- ## Worked examples > [!example] Example 1 — $\chi^2_5$ ka mean aur variance > $\chi^2_5$ ka $E$ aur $\operatorname{Var}$ nikalo. > **Step 1:** $E=k=5$. *Kyun?* Har $Z_i^2$ mean 1 contribute karta hai, 5 baar sum hota hai. > **Step 2:** $\operatorname{Var}=2k=10$. *Kyun?* Har $Z_i^2$ ka variance 2 hai, 5 independent terms pe sum hota hai. > [!example] Example 2 — sample variance se chi-squared > $N(\mu,\sigma^2)$ se $n=10$ ka sample deta hai $s^2 = 4$ true $\sigma^2=3$ ke saath. Test statistic ka distribution aur value kya hai? > **Step 1:** Statistic $\frac{(n-1)s^2}{\sigma^2}=\frac{9\cdot 4}{3}=12$. > **Step 2:** Distribution hai $\chi^2_{n-1}=\chi^2_9$. *Df=9 kyun?* $\bar x$ estimate karne ki cost ek df hai. > [!example] Example 3 — $t$ statistic banana > $n=16$, $\bar x = 52$, $s=8$, test $\mu=50$. > **Step 1:** $t = \dfrac{\bar x-\mu}{s/\sqrt n}=\dfrac{52-50}{8/4}=1$. > **Step 2:** Reference distribution $t_{15}$. *15 kyun?* $k=n-1=15$. > **Step 3:** Note karo ki hum $Z$ ki jagah $t$ use karte hain kyunki $\sigma$ $s$ se **estimated** hai. > [!example] Example 4 — $t^2$ as $F$ > Agar $t\sim t_{12}$, to $t^2$ kya hai? > **Step 1:** $t^2 = F_{1,12}$. *Kyun?* Numerator $Z^2$ hai $\chi^2_1$ (df 1) over $\chi^2_{12}/12$. > [!example] Example 5 — do variances ke liye F > Do samples: $s_1^2=10$ ($n_1=6$), $s_2^2=4$ ($n_2=9$), equal true variances. > **Step 1:** $F = s_1^2/s_2^2 = 10/4 = 2.5$. > **Step 2:** Distribution $F_{d_1,d_2}=F_{5,8}$. *Kyun?* $d_1=n_1-1=5$, $d_2=n_2-1=8$. --- > [!mistake] Common mistakes (steel-manned) > **1. Sample variance ke liye $n$ df use karna.** > *Ye sahi kyun lagta hai:* $n$ data points hain, to "$n$ pieces of information." > *Fix:* ek piece $\sum(x_i-\bar x)=0$ force karne mein use ho jaati hai. Free pieces $= n-1$. **Df = data − estimated parameters.** > > **2. $F$ mein har $\chi^2$ ko apne df se divide karna bhool jaana.** > *Ye sahi kyun lagta hai:* "$F$ sirf do chi-squareds ka ratio hai." > *Fix:* df se divide kiye bina, ratio ka scale $k$ pe depend karta hai. Divide karne se har part ka mean $\approx 1$ ho jaata hai isliye equal variances ke under $F\approx 1$. > > **3. Unknown $\sigma$ ke saath small samples ke liye $t$ ki jagah $Z$ use karna.** > *Ye sahi kyun lagta hai:* "$\bar x$ normal hai, to use standardise karo." > *Fix:* aapne $\sigma$ ko random $s$ se replace kiya, uncertainty badhti hai → fatter tails → $t_{n-1}$ use karna zaroori hai. > > **4. Ye sochna ki $t$ ke tails hamesha heavy rahenge.** > *Ye sahi kyun lagta hai:* "$t$ hamesha normal se alag hota hai." > *Fix:* Jab $k\to\infty$, $s\to\sigma$, denominator → 1, aur $t_k\to N(0,1)$. --- > [!recall]- Feynman: ek 12-saal ke bache ko explain karo > Socho tum measure kar rahe ho ki darts bullseye se kitna door girti hain. **Chi-squared** un *squared* misses ko add karta hai — ek "total wobble" score; zyada darts ($k$) matlab zyada expected score. **t** aisa hai jaise tum apni aim ki accuracy guess karo jab tumhe apni steadiness *pata nahi* — tum use kuch throws se estimate karte ho, isliye tum less sure ho, aur crazy outcomes zyada hoti hain (fat tails). **F** do players ki wobbliness ko ratio ke roop mein compare karta hai: agar yeh 1 ke paas hai, dono equally shaky hain. "Degrees of freedom" = kitne throws *truly free* the vary karne ke liye average compute karne ke baad. > [!mnemonic] Builders yaad rakho > **"Square, Scale, Split."** > - $\chi^2$ = normals ko **Square** karo aur add karo. > - $t$ = ek normal **Scaled** by estimated SD ($\sqrt{\chi^2/k}$). > - $F$ = ek chi-squared-per-df ko doosre se **Split** karo. > Aur **"df = data − constraints."** --- ## Connections - [[Standard Normal Distribution]] — wo atom jisse teeno bane hain. - [[Sample Variance and Bessel's Correction]] — $n-1$ df ka source. - [[Hypothesis Testing]] — t-tests, chi-squared goodness-of-fit, F-tests. - [[ANOVA]] — F between/within mean squares ke ratio ke roop mein aata hai. - [[Central Limit Theorem]] — kyun $\bar x$ normal hai, $t$ enable karta hai. - [[Gamma Distribution]] — $\chi^2_k$ hai $\text{Gamma}(k/2,\,1/2)$. --- ## #flashcards/maths $\chi^2_k$ ko normals se kaise define karte hain? ::: $\sum_{i=1}^k Z_i^2$ ke roop mein jahan $Z_i$ independent $N(0,1)$ hain. $\chi^2_k$ ka mean aur variance? ::: Mean $=k$, variance $=2k$. $E[\chi^2_k]=k$ kyun hai? ::: Kyunki $k$ independent terms mein se har ek ke liye $E[Z^2]=1$ hai. $(n-1)s^2/\sigma^2$ kaun sa distribution follow karta hai? ::: $\chi^2_{n-1}$. Sample variance ke liye $n-1$ df kyun? ::: $\bar x$ estimate karne se $\sum(x_i-\bar x)=0$ impose hoti hai, ek df cost hoti hai. $t_k$ ki definition? ::: $Z/\sqrt{V/k}$ jahan $Z\sim N(0,1)$, $V\sim\chi^2_k$, independent. $t$ ke tails $Z$ se fatter kyun hain? ::: Iska denominator $\sigma$ ka random estimate hai; chote estimates $t$ ko inflate karte hain. $k\to\infty$ par $t_k$ kya ban jaata hai? ::: Standard normal $N(0,1)$. $t_k$ ka variance? ::: $k/(k-2)$ for $k>2$. $F_{d_1,d_2}$ ki definition? ::: $(U/d_1)/(V/d_2)$ jahan $U\sim\chi^2_{d_1}$, $V\sim\chi^2_{d_2}$, independent. $t_k^2$ ka distribution kya hai? ::: $F_{1,k}$. $F$ ki reciprocal property? ::: $1/F_{d_1,d_2}\sim F_{d_2,d_1}$. $F$ mein har $\chi^2$ ko apne df se divide kyun karte hain? ::: Har ek ko mean $\approx 1$ pe normalise karne ke liye taaki equal variances ke under $F\approx 1$ ho. $F_{d_1,d_2}$ ka mean? ::: $d_2/(d_2-2)$ for $d_2>2$. Degrees of freedom ka general rule? ::: df = (data points ki number) − (estimated parameters/constraints ki number). ## 🖼️ Concept Map ```mermaid flowchart TD Z[Standard normal Z ~ N 0,1] CHI[Chi-squared with k df] T[Student t with k df] F[F distribution] DF[Degrees of freedom] CON[Constraint sum of deviations = 0] S2[Sample variance s squared] MV[Mean k, Variance 2k] Z -->|sum of k squared| CHI CHI -->|counts free directions| DF CON -->|removes one df| DF Z -->|numerator| T CHI -->|scaled denominator| T T -->|estimated variance| FT[Fatter tails than normal] CHI -->|ratio of two per df| F CHI -->|gives| MV S2 -->|n-1 s2 over sigma2| CHI CON -->|from estimating mean| S2 ``` ## 🔬 Deep Dive > [!intuition] Aur gehraai mein jaao — visual, zero se > Is topic ki step-by-step 3Blue1Brown-style breakdowns. - [[4.9.09 D1 Foundations|D1 · Foundations — har symbol zero se]] - [[4.9.09 D2 Visual Walkthrough|D2 · Visual walkthrough — derivation pictures mein]] - [[4.9.09 D3 Worked Examples|D3 · Worked examples — har scenario]] - [[4.9.09 D4 Exercises|D4 · Exercises — graded, full solutions]] - [[4.9.09 D5 Question Bank|D5 · Question bank — concept traps]]