z-test, t-test, chi-squared goodness of fit, F-test

Intuition The big picture

Every one of these tests asks the same question: "Is the difference I see in my data real, or could random noise alone have produced it?" We build a test statistic = (signal we measure) / (noise we expect). If that ratio lands in the extreme tail of a known reference distribution, the noise-only story becomes implausible and we reject the null hypothesis $H_0$ . The four tests differ only in what they compare and which reference distribution the noise follows:

z-test — a mean, when $\sigma$ is known (or huge $n$ ) → Normal.
t-test — a mean, when $\sigma$ is estimated from the sample → Student's $t$ .
chi-squared GOF — whether observed counts match expected counts → $\chi^2$ .
F-test — whether two variances are equal (ratio of variances) → $F$ .

1. The common machinery (WHY everything below works)

WHY a ratio? A raw difference like $\bar x - \mu_0 = 3$ means nothing until you ask "3 compared to what spread?" Dividing by the noise scale standardizes the difference into "number of standard errors," which is comparable across problems.

HOW the Central Limit Theorem powers the z/t tests. For a sample mean of size $n$ : $\mathbb{E}[\bar X]=\mu,\qquad \operatorname{Var}(\bar X)=\frac{\sigma^2}{n}$

Derivation of the variance (first principles): with i.i.d. $X_i$ , $\operatorname{Var}\!\left(\frac1n\sum X_i\right)=\frac1{n^2}\sum\operatorname{Var}(X_i)=\frac{1}{n^2}\,n\sigma^2=\frac{\sigma^2}{n}.$ So the standard error is $\mathrm{SE}=\sigma/\sqrt n$ . By the CLT, $\bar X\approx \mathcal N(\mu,\sigma^2/n)$ , which means $Z=\frac{\bar X-\mu_0}{\sigma/\sqrt n}\sim \mathcal N(0,1)\quad\text{under }H_0.$ This is the z-statistic — and it's the template for the rest.

2. z-test (mean, $\sigma$ known)

WHY $\sigma$ known is special: the denominator is a fixed constant, so the only randomness is in $\bar x$ , which is exactly Normal — no extra uncertainty leaks in.

3. t-test (mean, $\sigma$ unknown)

WHY divide by $n-1$ (Bessel's correction)? Deviations are taken from $\bar x$ , not the true $\mu$ . The sample is "closest" to its own mean, so $\sum(x_i-\bar x)^2$ systematically underestimates spread. We lose 1 degree of freedom (the constraint $\sum(x_i-\bar x)=0$ ), and dividing by $n-1$ makes $s^2$ unbiased: $\mathbb E[s^2]=\sigma^2$ .

Figure — z-test, t-test, chi-squared goodness of fit, F-test

4. Chi-squared goodness of fit

WHY divide by $E_i$ (first principles): for counts, the count in a cell is roughly Poisson with mean $E_i$ , so its variance is also $\approx E_i$ . Then $(O_i-E_i)/\sqrt{E_i}$ is a standardized $z$ -like quantity; squaring and summing $\nu$ of them gives a $\chi^2_\nu$ distribution.

WHY $\nu=k-1$ (with no extra params): the counts are constrained by $\sum O_i = \sum E_i = n$ (total fixed), removing 1 degree of freedom from the $k$ cells.

5. F-test (compare two variances)

WHY this is an F: each $s^2$ is (up to scale) a $\chi^2/\nu$ . The ratio of two independent $\chi^2$ 's each divided by their d.f. is, by definition, $F$ . So the F-test is "two chi-squareds racing."

Common mistake Steel-manned traps

"I'll use z because Normal is simpler, even with $n=6$ and unknown $\sigma$ ." Feels right: CLT, Normal everywhere. Fix: with unknown $\sigma$ and small $n$ , the estimated $s$ adds uncertainty → fatter-tailed $t$ . Using $1.96$ instead of $2.78$ rejects too often.

"Chi-squared cells with $E_i=2$ are fine." Feels right: it's just arithmetic. Fix: the $\chi^2$ approximation needs $E_i\gtrsim 5$ ; tiny expecteds inflate the statistic. Pool categories.

"Big p-value proves $H_0$ true." Feels right: failed to reject = accepted. Fix: you only failed to find evidence; absence of evidence ≠ evidence of absence.

"Forgetting to subtract estimated parameters in $\nu$ ." Feels right: $\nu=k-1$ always. Fix: every parameter you fit from the same data costs another d.f.: $\nu=k-1-m$ .

Recall Explain to a 12-year-old

Imagine you measure something and it looks a bit off from what you expected. Is it really off, or did you just get unlucky? You make a "weirdness score" = how far off you are ÷ how much wobble is normal. If the score is huge, "just unlucky" stops being believable, so you say something real is going on. The z and t scores check if an average is off (t is for when you're not sure how wobbly things are, so it's more cautious). Chi-squared checks if your tally of categories (like die faces) matches what it should. F checks if one thing wobbles more than another. Same trick, different shapes of "what luck can do."

Flashcards

When do you use a z-test vs a t-test for a mean?

z when population

\sigma

is known (or

n

huge); t when

\sigma

is unknown and estimated by

s

Why does the t-distribution have fatter tails than the Normal?

Because

s

in the denominator is itself a noisy estimate of

\sigma

, adding extra variability; controlled by

\nu=n-1

State the one-sample z-statistic.

Z=(\bar x-\mu_0)/(\sigma/\sqrt n)\sim\mathcal N(0,1)

under

H_0

Why divide by

n-1

in the sample variance?

Deviations are from

\bar x

not

\mu

; the constraint

\sum(x_i-\bar x)=0

removes one d.f., and

n-1

makes

s^2

unbiased.

Write the chi-squared GOF statistic and its d.f.

\chi^2=\sum (O_i-E_i)^2/E_i

\nu=k-1-m

(

m

= estimated parameters).

Why divide each chi-squared term by

E_i

Counts are ~Poisson with variance

\approx E_i

, so

(O_i-E_i)/\sqrt{E_i}

is standardized before squaring.

What does the F-test compare and what's its statistic?

Equality of two variances;

F=s_1^2/s_2^2\sim F_{n_1-1,\,n_2-1}

, larger variance on top.

Why is the ratio of variances an F distribution?

Each

s^2

is (scaled)

\chi^2/\nu

; ratio of two independent

\chi^2/\nu

is by definition F.

Rule of thumb for chi-squared GOF validity?

Each expected count

E_i \gtrsim 5

; otherwise pool categories.

Does a large p-value prove

H_0

No — you only failed to find evidence against it; it doesn't confirm

H_0

Connections

Central Limit Theorem — justifies Normality of $\bar X$ for z/t.
Normal Distribution — limiting case of $t$ as $\nu\to\infty$ .
Student's t-distribution and Chi-squared Distribution and F-distribution — the reference tails.
Degrees of Freedom — counts independent pieces of information.
Hypothesis Testing and p-value and Type I and Type II Errors.
ANOVA — generalizes the F-test to many group means.
Bessel's Correction — the $n-1$ unbiasedness argument.

Concept Map

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho, in chaaron tests ka core idea ek hi hai: hum ek "weirdness score" banate hain jo hota hai (jo difference dikha) divided by (jitna noise normally expect karte hain). Agar yeh score bahut bada ho jaye, matlab "sirf luck se hua hoga" wali kahani believable nahi rehti, to hum $H_0$ ko reject kar dete hain. Bas itni si baat hai — fark sirf yeh hai ki kya compare kar rahe ho aur reference distribution kaunsa hai.

z-test tab jab tum mean check kar rahe ho aur population ka $\sigma$ already pata ho — denominator fixed hai, to score seedha Normal $\mathcal N(0,1)$ follow karta hai. t-test tab jab $\sigma$ pata nahi, to tum $s$ (sample SD) se estimate karte ho. Lekin $s$ khud noisy hai, isliye extra uncertainty aati hai aur distribution thoda fat-tailed ho jata hai — yeh hai Student's $t$ , jiska shape degrees of freedom $\nu=n-1$ pe depend karta hai. Chhote sample mein zyada cautious rehna padta hai (1.96 ki jagah 2.78 type critical value).

Chi-squared goodness of fit categories ke liye hai: observed count $O_i$ vs expected $E_i$ . Har cell ka contribution $(O_i-E_i)^2/E_i$ — divide $E_i$ se isliye kyunki count ki variance bhi lagbhag $E_i$ hoti hai (Poisson logic). Sab add karo; bada total matlab model fit nahi ho raha. F-test do variances ka ratio $s_1^2/s_2^2$ leta hai — agar dono truly equal hain to ratio 1 ke aaspaas hona chahiye; door chala gaya to variances different hain.

Yaad rakhne ka mantra: Z Sure, T Tries, Chi Counts, F Fights. Aur ek important baat — bada p-value ka matlab yeh nahi ki $H_0$ sach hai; bas tumhe uske against evidence nahi mila. Exam mein degrees of freedom mein estimated parameters subtract karna mat bhoolna!

z-test, t-test, chi-squared goodness of fit, F-test

1. The common machinery (WHY everything below works)

2. z-test (mean, $\sigma$ known)

3. t-test (mean, $\sigma$ unknown)

4. Chi-squared goodness of fit

5. F-test (compare two variances)

Flashcards

Connections

Concept Map

Hinglish (regional understanding)

Go deeper — visual, from zero

Test yourself — Probability Theory & Statistics

Connections