Visual walkthrough — Hypothesis testing — null - alternative, test statistic, p-value, errors (Type I & II)

Here is the whole plan in one glance. Read it, then we build every box.

Step 1 — One measurement is a noisy dart

WHAT. We have a machine that is supposed to fill each bottle to some target amount. We call one measured fill $X_1$. Measure another and we get $X_2$, and so on up to $X_n$, where $n$ is just "how many bottles we checked" (a plain counting number like $64$).

Each single measurement is a noisy dart: even a perfect machine never pours exactly the same amount twice. Two invented words describe that noise:

WHY these two. A dartboard is fully summarised by where the cloud sits ($\mu$) and how wide it is ($\sigma$). Nothing else about the noise matters for what follows.

PICTURE. The scattered pale-blue darts below have a centre (yellow line $\mu$) and a spread (the pink band $\pm \sigma$).

Step 2 — Averaging many darts calms the wobble

WHAT. Instead of trusting one noisy dart, we average all $n$ of them into a single number, the sample mean:

$\bar{X}=\underset{\text{share equally}}{\underbrace{\frac{1}{n}}}\underset{\text{add up all }n\text{ fills}}{\underbrace{\sum _{i=1}^n{X}_i}}.$

Read $\bar{X}$ as "X-bar." The bar means "averaged." The symbol ${\sum }_{i=1}^n{X}_i$ is a stack of instructions: start at $i=1$, add $X_i$, step $i$ up by one, stop after $i=n$ — i.e. $X_1+X_2+\cdots +X_n$.

WHY average. One dart is jumpy; the average of many darts barely moves, because a high fill here tends to cancel a low fill there. Averaging is our noise-reducer.

PICTURE. Left: single darts, wide scatter. Right: their averages $\bar{X}$, a tight cluster hugging the same centre. Same centre, smaller spread — that is the whole point.

Step 3 — Exactly how much calmer? The variance shrinks by $n$

WHAT. We now measure how much the averaging calmed things down. Two facts, each earned:

$\mathbb{E}[\bar{X}]=\mu ,\mathrm{Var}(\bar{X})=\frac{{\sigma }^2}{n}.$

• $\mathbb{E}[\bar{X}]=\mu$: the average of the averages sits on the same centre. Averaging does not move the target, it only sharpens it.
• $\mathrm{Var}(\bar{X})=\frac{{\sigma }^2}{n}$: the spread of $\bar{X}$ is the single-dart spread ${\sigma }^2$ divided by $n$.

WHY the $\frac{1}{n}$. For independent darts, variances of a sum add: $\mathrm{Var}(\sum X_i)=n{\sigma }^2$. But $\bar{X}$ multiplies that sum by the constant $\frac{1}{n}$, and a constant $c$ pulls out of variance squared: $\mathrm{Var}(cY)=c^2\mathrm{Var}(Y)$. So

$\mathrm{Var}(\bar{X})=(\frac{1}{n}{)}^2\cdot n{\sigma }^2=\frac{{\sigma }^2}{n}.$

The $\frac{1}{n^2}$ from squaring the constant beats the $n$ from adding — net shrink by $n$. This is the seed of the Central Limit Theorem and of Standard Error.

PICTURE. Three bell shapes for $n=1,4,16$: same centre $\mu$, each half as wide as $n$ quadruples (because width $\propto 1/\sqrt{n}$).

Step 4 — Name the wobble: standard error

WHAT. The spread of $\bar{X}$ in original units deserves its own name — the standard error:

$\mathrm{SE}=\sqrt{\mathrm{Var}(\bar{X})}=\frac{\sigma }{\sqrt{n}}.$

• $\sigma$ = spread of one dart (millilitres).
• $\sqrt{n}$ = how much averaging tightened it.
• $\mathrm{SE}$ = spread of $\bar{X}$ (also millilitres) — our yardstick.

WHY square-root. Variance lives in squared units (ml${}^2$); to compare against a mean in ml we must take the square root to return to ml. See Standard Error for more.

PICTURE. The wide dart-cloud (spread $\sigma$) beside the narrow average-cloud (spread $\mathrm{SE}$), with a labelled bracket showing $\mathrm{SE}=\sigma /\sqrt{n}$.

Step 5 — Assume the boring story, then measure the gap

WHAT. The "boring story" is a specific claimed centre, ${\mu }_0$ (mu-nought) — e.g. "the machine really pours $500$ ml on average." This is the null hypothesis $H_0:\mu ={\mu }_0$. We assume it true and look at the gap between what we saw and what it predicts:

$\text{raw gap}=\bar{X}-{\mu }_0.$

• $\bar{X}$ = what we measured.
• ${\mu }_0$ = what $H_0$ promised.
• their difference = how far off the boring story is (in ml).

WHY assume $H_0$. We need a fixed centre to measure against. The boring claim gives us that fixed pin; everything is judged relative to it. We are not saying $H_0$ is true — we are asking "if it were, how surprising is this gap?"

PICTURE. The bell of $\bar{X}$ centred at ${\mu }_0$ (the assumption), with our observed $\bar{X}$ marked off to the side and the raw gap drawn as a yellow arrow.

Step 6 — Measure the gap in yardsticks: the z-statistic

WHAT. A raw gap of "$3$ ml" is meaningless until we know whether $3$ ml is a lot or a little. So we divide the gap by the yardstick $\mathrm{SE}$:

$\boxed{Z=\frac{\stackrel{\text{gap in ml}}{\overbrace{\bar{X}-{\mu }_0}}}{\underset{\text{one yardstick in ml}}{\underbrace{\sigma /\sqrt{n}}}}}$

• Numerator $\bar{X}-{\mu }_0$: the raw gap.
• Denominator $\sigma /\sqrt{n}=\mathrm{SE}$: one standard error.
• $Z$: how many standard errors the data sits from $H_0$ — a pure number, no units.

WHY divide. Dividing a length by the same-unit yardstick cancels the units and answers "how many wobbles away am I?" $Z=-3$ means "three standard errors below the claim." Because of the Central Limit Theorem, this $Z$ follows the standard normal $\mathcal{N}(0,1)$ — a fixed bell we can read probabilities off. (When $\sigma$ is unknown and we substitute the sample spread $s$, the same recipe gives $T$ on the fatter-tailed Student t-distribution.)

PICTURE. The standard normal bell on a ruler marked in standard errors $-3,-2,-1,0,1,2,3$, with $\bar{X}$'s raw gap being "measured" by stepping off three yardsticks to land at $Z=-3$.

Step 7 — Read the surprise: shaded tail = p-value

WHAT. The p-value is the shaded area in the tail(s) beyond our $Z$ under the standard bell:

$p_{\text{two-tailed}}=2P(Z\ge |z_{\text{obs}}|).$

• $|z_{\text{obs}}|$: how far out we landed, ignoring sign.
• $P(Z\ge \cdot )$: chance a fair (null-true) world coughs up something at least this extreme in one tail.
• factor $2$: for "$\ne$" tests, extreme means either side, so shade both tails.

WHY area, why two tails. Area = probability. Since $H_1:\mu \ne {\mu }_0$ counts a suspiciously high fill as damning as a suspiciously low one, we shade both ends. A right-tailed test ($>$) shades one end; a left-tailed test ($<$) shades the other. Choosing the tail after peeking at the data is cheating — fix it beforehand.

PICTURE. The bell with both tails past $\pm 3$ shaded pink; the tiny shaded area is $p\approx 0.0027$.

Step 8 — Edge & degenerate cases (never skip these)

WHAT / WHY / PICTURE for each corner the reader could hit:

• $Z=0$ exactly ($\bar{X}={\mu }_0$): the data sits on the claim. $p$ is as large as it gets ($p=1$ two-tailed). Zero surprise — never reject.
• $n=1$: $\mathrm{SE}=\sigma /\sqrt{1}=\sigma$. One dart, no averaging help — the widest possible bell. The method still runs, it is just weak.
• $n\to \infty$: $\mathrm{SE}=\sigma /\sqrt{n}\to 0$. The bell collapses to a spike; any nonzero gap gives a giant $|Z|$ and rejects. A huge sample flags even a tiny, meaningless difference — statistical significance is not practical importance.
• $\sigma$ unknown: swap in sample spread $s$, use $T$ on the Student t-distribution with $n-1$ degrees of freedom — fatter tails to pay for our uncertainty about the spread. For small $n$ this makes rejecting harder, as it should.

The one-picture summary

Everything on one board: darts → average → shrink by $n$ → yardstick $\mathrm{SE}$ → count yardsticks to $Z$ → shade the tail for $p$.

Active Recall