Foundations — Hypothesis testing and p-values

Before you can trust a single line of the parent note, you must be able to read it. This page lists every symbol and idea the parent uses, in build-order, so that no notation ever appears before you own it. A smart 12-year-old should be able to walk from top to bottom without ever getting stuck.

0. The picture we keep coming back to

Almost everything in hypothesis testing lives on one bell-shaped curve. Get comfortable with this picture now and the rest is labelling.

Everything below is a tool for drawing this curve and pointing at a spot on it.

1. Probability $P(\cdot )$ — the language of "how often"

The picture: think of a big jar of outcomes. $P(\text{heads})$ is the fraction of the jar that is "heads". If half the jar is heads, $P(\text{heads})=0.5$.

Why the topic needs it: the whole p-value is a probability — "how often would I see data this weird?" You cannot read a p-value without first being fluent in $P$.

The bar $\mid$ (given that): $P(A\mid B)$ reads "the probability of $A$, given that $B$ is already true." The vertical bar means "assuming...".

2. Random variable $X$ — a number that comes out of chance

The picture: a slot machine. You pull the lever (run the experiment) and a number pops out. $X$ is the name of that number-generator; the value it lands on this time is a plain number like $65$.

Why the topic needs it: the test statistic (next few sections) is a random variable. To ask "how weird is 65 heads," you must imagine $X$ could have been many values, each with some probability. See 1.3.1-Random-variables-and-distributions.

3. Distribution — the full menu of possible values and their chances

The picture: the bar chart above. Each bar is one possible number of heads; tall bars = likely, short bars = rare. The bars for a fair coin form the bell shape from Section 0.

Two named distributions the parent uses:

Why the topic needs it: the p-value is an area under the distribution. No distribution, nowhere to measure area.

4. Mean $\mu$ and standard deviation $\sigma$ — center and spread

The picture: $\mu$ is where the curve peaks (the middle). $\sigma$ is the horizontal distance from the peak out to where the curve bends from "falling steeply" to "flattening out." A skinny bell has small $\sigma$; a fat bell has large $\sigma$.

Why the topic needs it: to say a result is "far from expected," you need a center to measure from ($\mu$) and a ruler to measure with ($\sigma$).

5. Sample mean $\bar{X}$ vs. true mean ${\mu }_0$ — measured vs. claimed

The picture: two dots on the number line. ${\mu }_0$ is the target the null hypothesis paints. $\bar{X}$ is where your actual result landed. Hypothesis testing measures the gap between them.

Why the topic needs it: every test statistic is essentially $\text{(what I saw)}-\text{(what was claimed)}$, i.e. $\bar{X}-{\mu }_0$.

6. Test statistic — the single number we shade against the curve

The picture: all your messy data gets crushed down to one dot on the horizontal axis of the bell. That dot is the test statistic. The p-value is then just the tail area beyond that dot.

Why the topic needs it: you can't shade "your data" against a curve — data isn't a single point. The test statistic is the bridge that turns a whole dataset into one shadeable position.

7. The z-score $Z$ — "how many rulers away?"

Worked reading: $Z=\frac{65-50}{5}=3$. You are 3 standard deviations to the right of the boring center — deep in a skinny tail.

All the cases (so you never get surprised):

• $Z=0$: result sits exactly at expected. Totally boring.
• $Z>0$: result is above expected (right side of the bell).
• $Z<0$: result is below expected (left side of the bell).
• $|Z|$ large (like 3): far out in a tail → surprising.
• $|Z|$ small (like 0.4): near the fat middle → unsurprising.

The bars $|\cdot |$ mean absolute value — throw away the sign, keep the size. $|-3|=3$. We use it because "3 steps left" and "3 steps right" are equally surprising; we care about distance, not direction.

This standardizing trick is the heart of 1.3.15-Central-limit-theorem and 1.3.18-Confidence-intervals.

8. Tails, and the p-value as an area

The picture (above): your $Z$ marks a spot. Shade everything further out than that spot. That shaded area is the p-value.

9. The Greek gatekeepers: $\alpha$, $\beta$, and the hypotheses

The picture: $\alpha$ is a fence painted on the bell's tails. If your p-value area is smaller than $\alpha$, you're outside the fence → reject $H_0$.

10. The extras the parent leans on

How it all feeds the topic

Equipment checklist

Test yourself — you should be able to say each answer out loud before reading the parent note.