Hypothesis testing and p-values: Visual walkthrough

This page is the visual companion to the parent topic. We assume you have never seen a bell curve, a $Z$ -score, or the word "tail." We earn each one.

Step 1 — Draw every result a fair coin could give

WHAT. Before touching our data, we ask: if the coin really were fair, what results are even possible in 100 flips? Let $X$ stand for the number of heads in those 100 flips. $X$ could be $0, 1, 2, \dots$ , up to $100$ . Some counts are common (around 50), some are almost impossible (like 0 or 100).

WHY. Hypothesis testing works by assuming the "boring" story is true — the coin is fair — and asking how our data looks against that story. So the first thing we need is the full landscape of possible values of $X$ under fairness. Only then can we say whether 65 is normal or bizarre. This landscape is a probability distribution.

PICTURE. Each bar below is one possible head-count. The height of a bar is the probability that $X$ equals exactly that many heads if the coin is fair.

Step 2 — The hill is (almost) a bell curve

WHAT. That staircase of bars, for large $n$ , has a smooth twin: a bell curve (the normal distribution). We swap the jagged bars for the smooth bell.

WHY. Two reasons. (1) The bell curve has a clean formula, so we can compute areas with a table instead of adding up 100 binomial bars. (2) The Central Limit Theorem guarantees this swap is accurate: since $X$ is a sum of 100 tiny independent $0$ -or- $1$ flips, the total always looks like a bell. That is why the bell curve is the right tool here and not some arbitrary curve.

PICTURE. The smooth mint bell lies almost exactly on top of the lavender bars.

Formula The bell curve's actual formula (the normal PDF)

The smooth mint curve is not just "some hill" — it has one exact equation. Its height at a value $x$ (a head-count) is $f (x) = fixes total area to 1 \frac{1}{σ 2 π} drops off as you leave the peak e^{- \frac{1}{2} (\frac{x - μ}{σ})^{2}}$

$μ$ is where the peak sits, $σ$ is how wide it is (both pinned down just below).
$\frac{x - μ}{σ}$ is the distance from the peak measured in widths — the very quantity we'll name $Z$ in Step 4. When it's $0$ (at the peak) the exponent is $0$ and $f$ is largest; the further out, the more negative the exponent, so the curve dives toward zero — that's the bell shape.
$e$ is Euler's number ( $\approx 2.718$ ); raising it to a negative square makes a smooth symmetric hump.
The front factor $\frac{1}{σ 2 π}$ is just the scaling that makes the whole area under the curve equal exactly 1 (total probability must be 1). Wider bell → shorter, so area stays 1.

This mint curve is what every shaded area from Step 5 onward is measured under.

To pin the bell down we need just two numbers: where its peak sits (the mean $μ$ ) and how wide it is (the standard deviation $σ$ ).

Step 3 — Place our real data on the picture

WHAT. We drop a vertical line at $X = 65$ — the value of $X$ (the observed head count) we actually got.

WHY. The distribution is the world of "what could happen if fair." Our data is a single value of $X$ in that world. To judge weirdness we must see where it lands relative to the hill.

PICTURE. The coral line at 65 sits far out on the right slope, past the bulk of the bell.

By eye it already looks unusual — but "looks unusual" isn't a number. Step 4 turns the distance from the peak into a clean, unit-free score.

Step 4 — Measure the distance in "widths": the Z-score

WHAT. We measure how far $X = 65$ is from the peak 50, counted in standard deviations (in units of $σ$ ). Call that count $Z$ .

WHY. Saying "65 is 15 heads above average" is meaningless on its own — 15 heads is huge for a narrow bell and tiny for a wide one. Dividing by the width $σ$ makes the distance comparable across any problem. A $Z$ of 3 always means "3 widths out," whether we flip coins, measure heights, or test models. This is exactly the standardizing move used in confidence intervals.

PICTURE. The horizontal axis is now relabelled in widths: 0 at the peak, and 65 sitting at $Z = 3$ .

Step 5 — "As extreme or more": why we shade a tail

WHAT. The p-value is not the probability of getting exactly 65. It's the probability of getting a result at least as weird as 65. So we shade the whole tail — every outcome $Z \geq 3$ — not just the single line.

WHY. Any exact outcome (even the most likely one) has a small probability, so "probability of exactly 65" would be a bad surprise-meter. What we truly care about is: how often would a fair coin surprise us this much or more? That's an area under the $N (0, 1)$ tail, from our line outward to the edge.

PICTURE. Everything from $Z = 3$ rightward is shaded coral — the "at least this weird" region.

The area of that shaded right tail under the standard normal is a known number: $P (Z \geq 3) \approx 0.00135$

Step 6 — Both directions: the two-tailed p-value

WHAT. Our alternative was $H_{1} : p \neq = 0.5$ — the coin could be biased either way. So "as weird as 65 heads" also includes its mirror image: 35 heads ( $Z = - 3$ ), an equally strong bias toward tails. We shade both tails and add their areas.

WHY. We asked a two-sided question ("is it biased at all?"), so a result that's just as extreme in the tails direction should count as equally surprising. Ignoring the left tail would secretly turn our question into a one-sided one. Because the standard normal is symmetric, the left tail has the same area as the right, so we simply double.

PICTURE. Two symmetric coral tails, one at $Z = + 3$ and its twin at $Z = - 3$ .

Reading it in words: if the coin were perfectly fair, we'd see a result this extreme only about 0.27 % of the time. That's rare.

Step 7 — The verdict: compare to the threshold $α$

WHAT. We pick a line-in-the-sand before looking — the significance level $α$ , usually $0.05$ — and reject the fair-coin story only if our p-value falls below it.

WHY. We need a rule agreed on in advance so we can't move the goalposts after seeing the data. $α = 0.05$ means "I'll accept being fooled by luck 5 % of the time." Choosing $α$ is really choosing how often you tolerate a false alarm — a Type I error.

PICTURE. A number line: the tiny p-value $0.0027$ sits far to the left of the $α = 0.05$ gate. Inside the gate → reject.

$p = 0.0027 < 0.05 = α ⟹ reject H_{0}$

Conclusion: strong evidence the coin is not fair.

Step 8 — Edge and degenerate cases (what the picture does at the extremes)

WHAT. We check the corners so no scenario surprises you.

WHY. A derivation you trust must survive its own limits.

PICTURE. Four mini-panels: data at the peak, data at a mild bulge, a one-tailed shade, and the "wall" of $p$ never reaching 0.

Data exactly at the peak ( $X = 50$ , $Z = 0$ ). Both tails are the whole bell, so $p = 2 \cdot 0.5 = 1$ . The largest possible p-value is $1$ — perfectly ordinary data can never be "weird," so we never reject. ✔
Mildly off-centre ( $X = 54$ , $Z = 0.8$ ). Tails are fat, $p \approx 0.42 > 0.05$ → fail to reject. Being a little above 50 is easily explained by luck.
One-tailed question. If $H_{1}$ were "biased toward heads" only, we shade the right tail alone: $p = P (Z \geq 3) \approx 0.00135$ , half the two-tailed value. Same data, different question, different p-value — so always fix your hypothesis before computing.
p can never hit 0. No matter how extreme the data, the standard-normal tail has some area — the coral never fully vanishes. So we can reject fairness but never disprove it with total certainty.

The one-picture summary

Every step, compressed: the fair-coin bell (Step 2), our data line at $Z = 3$ (Step 4), the two shaded tails (Step 6), and the verdict $p = 0.0027 < 0.05$ (Step 7) — all on one canvas.

Recall Feynman retelling — say it to a friend

A fair coin, flipped 100 times, usually lands near 50 heads — the exact chances come from the binomial formula $(k 100) 0. 5^{100}$ , and I can draw them as a bell-shaped hill centred on 50, about 5 heads wide. I flipped mine and got $X = 65$ . On my hill, 65 is three "widths" out to the right — I mark it with a line, and after re-centring to 0 and rescaling the width to 1, that spot is called $Z = 3$ on the standard bell (mean 0, width 1). Now the key move: I don't ask "what's the chance of exactly 65?" (silly — every exact number is rare). I ask "what's the chance a fair coin surprises me this much or more?" — that's the shaded area in the far-right tail, plus its mirror in the left tail because a bias toward tails would surprise me just as much. Those two slivers together cover about 0.27 % of all fair-coin worlds. Since I agreed beforehand to only cry "unfair!" when the surprise is below 5 %, and 0.27 % is way below, I reject the fair-coin story. I haven't proven bias — the tail never quite reaches zero, so certainty is off the table — but fairness is now a lousy explanation for what I saw.

Recall Quick self-test

What does the height of one bar in Step 1 represent, and what formula gives it? ::: The probability $P (X = k) = (k n) p^{k} (1 - p)^{n - k}$ that the head count $X$ equals exactly $k$ if the coin is fair. What is the formula for the smooth bell curve, and what does its front factor do? ::: $f (x) = \frac{1}{σ 2 π} e^{- \frac{1}{2} ((x - μ) / σ)^{2}}$ ; the front factor $\frac{1}{σ 2 π}$ scales the curve so its total area equals 1. Why divide by $σ$ to get $Z$ ? ::: To measure distance from the peak in widths, making it comparable across any problem — and it turns the curve into the standard normal $N (0, 1)$ . What distribution does $Z$ follow under $H_{0}$ ? ::: The standard normal distribution, mean $0$ and standard deviation $1$ , $Z \sim N (0, 1)$ . Why do we shade a whole tail, not a single line? ::: The p-value is the chance of data at least as extreme, which is an area, not a single-point probability. Why the factor of 2 in the two-tailed p-value? ::: Because $H_{1} : p \neq = 0.5$ counts extremes in both directions; the symmetric bell makes the two tails equal, so we double one. When is the normal approximation safe? ::: When both $n p \geq 10$ and $n (1 - p) \geq 10$ ; otherwise use the exact binomial. What's the largest a p-value can be, and when? ::: $1$ — when the data sits exactly at the peak ( $Z = 0$ ), the most ordinary possible result.

Related building blocks: 1.3.1-Random-variables-and-distributions · 1.3.15-Central-limit-theorem · 1.3.18-Confidence-intervals · 1.3.21-Type-I-and-Type-II-errors · 3.2.12-Multiple-testing-correction · 2.5.7-Statistical-significance-in-experiments