Foundations — Hypothesis testing — null - alternative, test statistic, p-value, errors (Type I & II)

Before you can read the parent note, you must own every symbol it throws at you. We build them one at a time, from absolute zero, each resting on the one before. Nothing is used before it is pictured.

1. Population vs sample — the two worlds

The whole subject exists because we are stuck peeking at a small sample but want to say something about the giant population behind it.

Look at the figure: the blue cloud is the huge population; the yellow dots we scooped out are the sample. Two different scoops give two different yellow patterns even though the blue cloud never changed — that wobble is the randomness we must tame.

WHY the topic needs this: every claim ("the mean is 500 ml") is about the blue world, but every number we compute comes from the yellow world. The gap between them is the entire problem.

2. $n$ — the sample size

Picture: count the dots. That's $n$. Bigger $n$ = you scooped more = your yellow snapshot is more trustworthy. This single number quietly controls both error rates later, so keep an eye on it. So from now on, "a sample of $n$ measurements" just means "$n$ yellow dots."

WHY flag this now: every result below — the variance of the average, the Central Limit Theorem, the whole standard-error machinery — silently relies on i.i.d. If the draws were correlated or came from different clouds, the formulas break. We state it once, loudly, here.

3. The mean $\mu$ and the sample mean $\bar{X}$

Two flavours, and mixing them up is the #1 beginner trap:

$\bar{X}=\frac{1}{n}{\sum }_{i=1}^n{X}_i$

So $\bar{X}=\frac{1}{n}(X_1+\cdots +X_n)$ is "add every measurement, split evenly." That's the yellow balance point.

WHY the topic needs it: $\mu$ is the thing the null hypothesis makes a claim about; $\bar{X}$ is the evidence we weigh against that claim.

4. Spread: variance ${\sigma }^2$ and standard deviation $\sigma$

Knowing the balance point isn't enough — we need to know how spread out the cloud is. A tight cloud and a fat cloud can share the same mean but tell very different stories.

In the figure both curves balance at the same $\mu$, but the red one is fat ($\sigma$ large — very spread) and the green one is skinny ($\sigma$ small — tightly bunched). $\sigma$ is literally the typical distance from a point to the mean.

WHY square then square-root? WHAT we did: squared distances so pluses and minuses can't cancel to a fake zero. WHY the square root at the end: to return to the original units so "$\sigma =8$ ml" is meaningful. This tool answers the question "how wide is the wobble?" — nothing else measures spread in the right units.

5. The Normal distribution — the bell

The height of the curve above a value tells you how relatively common that value is. Area under a slice = probability of landing in that slice. Total area = 1 (something must happen).

WHY the topic leans on it: thanks to the Central Limit Theorem (next section), the sample mean $\bar{X}$ follows this bell almost regardless of the raw data's shape. A known shape means we can compute exact probabilities — the raw fuel for p-values. See Normal Distribution for the full story.

6. The Central Limit Theorem — why the bell appears for free

Top panel: the raw population is a lopsided, un-bell-like mess. Bottom panel: the distribution of the sample means is a clean symmetric bell centred on $\mu$. This is the magic that lets us use Normal-curve maths even when the data itself is not Normal. Deep dive: Central Limit Theorem.

WHY the topic needs it: it is the licence to standardize $\bar{X}$ into a $z$ score and read probabilities off the bell. Without it, Step 3 of the parent's derivation is illegal. (The licence is only valid when the draws are i.i.d. — see §2.)

7. Standard error $\mathrm{SE}$ — the spread of the average

Here is the subtle jump. A single measurement wobbles by $\sigma$. But the average of $n$ measurements wobbles less, because lucky-high and lucky-low points partly cancel.

WHAT it means: SE is the standard deviation of $\bar{X}$ itself — how much your sample mean jitters if you re-scooped. WHY $\sqrt{n}$ and not $n$: for independent draws (§2) variances add, so the variance of the average is $\mathrm{Var}(\bar{X})={\sigma }^2/n$; taking the square root to get back to an SD gives $\sigma /\sqrt{n}$. WHAT it looks like: quadruple your sample and the average's wobble halves — diminishing returns. Full picture in Standard Error.

WHY the topic needs it: SE is the ruler for the test statistic. We measure "how far is $\bar{X}$ from the claim" not in ml, but in number of SEs — a universal, unit-free scale.

8. The claim itself: $H_0$ and the null value ${\mu }_0$

Before we can measure "how far from the claim," we need a symbol for the claim.

Picture: ${\mu }_0$ is a flag planted on the number line where the boring world says the true mean sits. Everything the test does is measure the distance from our evidence $\bar{X}$ to that flag.

WHY the topic needs it: $H_0$ is the defendant assumed innocent; ${\mu }_0$ is the precise story we test the data against. Without a fixed ${\mu }_0$ there is no "distance from the claim" to compute.

9. The z-score — distance measured in standard errors

Picture: relabel the bell's horizontal axis from "ml" into "steps of one SE." Now $z=-3$ means "three standard errors below the claimed mean" — impressively far, in any problem, for any units. That comparability is exactly why we standardize.

WHY the topic needs it: the test statistic $Z=\frac{\bar{X}-{\mu }_0}{\sigma /\sqrt{n}}$ is precisely this z-score applied to the sample mean $\bar{X}$, using the claimed centre ${\mu }_0$ (§8) and the ruler SE (§7).

10. When $\sigma$ is unknown: $s$, the Bessel correction, and the t-statistic

In real life we rarely know the population's true $\sigma$. So we estimate it from the sample and call the estimate $s$ (the sample standard deviation).

Using a guessed spread adds extra uncertainty, so the bell gets slightly fatter tails. That fatter bell is the Student t-distribution, written $t_{n-1}$. More data ⇒ more degrees of freedom ⇒ the t-bell tightens back toward the Normal. Full treatment: Student t-distribution.

WHY the topic needs it: this powers every test where $\sigma$ is unknown (parent's Worked Examples 2 and 3).

11. Probability symbols: $P(\cdot )$, $\alpha$, $\beta$, and $\mid$

The last cluster of notation — small but load-bearing.

The p-value is $P(\text{data this extreme}\mid H_0\text{ true})$ — read it as "probability of the evidence given the boring world $H_0$ (§8)," never the reverse. Flipping the bar is the prosecutor's fallacy the parent warns about.

Prerequisite map

Each arrow says "you cannot understand the target until you own the source." All roads feed the final box — the parent topic.

Equipment checklist

Self-test: can you say each aloud before revealing?