Hypothesis testing and p-values: Worked examples

This page is a worked-example gauntlet. The parent note showed you the machinery once. Here we run it through every shape a problem can take: one tail, two tails, the boundary where the p-value exactly equals $α$ , a degenerate "zero difference" input, a limiting "huge sample" case, a real-world word problem, and an exam-style trap. If you meet a case here, you have already seen it.

Before we start, one promise: every symbol used below is either defined in the parent or re-earned here. When a number pops out, it is checked in the verify block.

The scenario matrix

Think of a hypothesis test as a machine with knobs. The knobs are: which tail(s) we look at, the sign of the observed effect, and how extreme the input is. The table below lists every knob-setting we must cover so nothing surprises you later.

Cell	Case class	What makes it tricky	Covered by
A	Two-tailed, positive effect	must double the tail; direction doesn't matter	Ex 1
B	Two-tailed, negative effect	$z$ is negative — do we flip a sign?	Ex 2
C	One-tailed, right side	only "bigger" counts; don't double	Ex 3
D	One-tailed, wrong direction	effect goes the opposite way you hoped	Ex 4
E	Boundary case: p-value $= α$	reject or not? the " $<$ vs $\leq$ " edge	Ex 5
F	Zero / degenerate input	observed = null exactly, $z = 0$	Ex 6
G	Limiting case: huge $n$	tiny effect becomes "significant"	Ex 7
H	Real-world word problem	you must build $H_{0}, H_{1}$ yourself	Ex 8
I	Exam twist: given a target p-value, find the cutoff	run the machine backwards	Ex 9

Three definitions we lean on throughout, re-stated in plain words:

Definition The vocabulary, re-earned

Null hypothesis $H_{0}$ : the boring "nothing is going on" claim we assume true while we compute.
$π$ (true proportion), $π_{0}$ (its null value), $\overset{π}{^}$ (sample estimate): $π$ is the real chance we can't see; $π_{0}$ is the number $H_{0}$ claims it equals; $\overset{π}{^}$ is the fraction we actually observed.
$X$ (count of successes): the raw number of "yes" outcomes we counted (e.g. number of heads). It is a whole number between $0$ and $n$ .
$Z$ vs $z$ : capital $Z$ is the random variable — the whole bell-shaped spread of statistics we could get if we repeated the experiment. Lowercase $z$ is one fixed observed number — the value $Z$ took on for our actual data. So " $P (Z \geq z)$ " reads: "the chance the random $Z$ lands at or beyond our one observed $z$ ."
$SE$ (standard error): the standard deviation of a sample estimate — how much the estimate would wobble from sample to sample. For a proportion, $SE = π_{0} (1 - π_{0}) / n$ . It is the "width" we divide by when standardizing.
$z$ (the test statistic): "how many standard errors is my observation from what $H_{0}$ predicts?" A ruler measuring surprise, computed as $z = (observed - null) / SE$ .
p-value: the probability, if $H_{0}$ were true, of a result at least this surprising. A small p-value = my data is a bad fit for the boring story.
$α$ (significance level): the line in the sand. If the p-value $< α$ we reject $H_{0}$ . Usually $α = 0.05$ .

The picture below is the whole game: a bell curve (the distribution of the random variable $Z$ under $H_{0}$ ), your one observed value $z$ marked on it, and the shaded tail area = the p-value.

Figure — Hypothesis testing and p-values

Everything from here is: find $z$ , shade the right region, read the area. See 1.3.15-Central-limit-theorem for why $Z$ is bell-shaped at all, and 1.3.1-Random-variables-and-distributions for what "random variable" and "distribution" mean.

Cell A — Two-tailed, positive effect

Worked example Ex 1 · A coin gives too many heads

Flip $n = 100$ times, and let $X$ be the count of heads (the successes). We observe $X = 64$ . Test $H_{0} : π = 0.5$ vs $H_{1} : π \neq = 0.5$ .

Forecast: more heads than the expected 50, so $z > 0$ . Two-tailed means we double one tail. Guess: is the p-value above or below $0.05$ ?

Mean and spread under $H_{0}$ . With $π_{0} = 0.5$ : $μ = n π_{0} = 100 (0.5) = 50$ , $σ = n π_{0} (1 - π_{0}) = 25 = 5$ . Why this step? These are the centre and width of the bell the count $X$ would follow if the coin were fair — the yardstick for "normal."
Standardize. $z = \frac{X - μ}{σ} = \frac{64 - 50}{5} = \frac{14}{5} = 2.8$ . Why this step? Turns "64 heads" into "2.8 std-devs above centre," a scale we can look up.
Both tails. $p-value = 2 \cdot P (Z \geq 2.8) = 2 (0.002555) = 0.00511$ . Why this step? $H_{1}$ says "biased either way," so a symmetric result of 36 heads would be equally weird — we count both tails. Read " $P (Z \geq 2.8)$ " as "chance the random $Z$ exceeds our observed $z = 2.8$ ."
Decide. $0.00511 < 0.05 \Rightarrow$ reject $H_{0}$ .

Verify: $z = 2.8$ sits well past the "2 std-dev" rule-of-thumb (which gives a p-value $\approx 0.05$ ), so a smaller p-value is sanity-consistent. Units: $z$ is dimensionless ✓.

Cell B — Two-tailed, negative effect

Worked example Ex 2 · A coin gives too FEW heads

Same coin, $n = 100$ , count of heads now $X = 36$ . Same two-tailed test.

Forecast: fewer heads → $z < 0$ . Does the negative sign change the p-value? Guess before reading.

Standardize. $z = \frac{X - μ}{σ} = \frac{36 - 50}{5} = \frac{- 14}{5} = - 2.8$ . Why this step? Same yardstick; the sign just tells us we're on the left of centre.
Take the magnitude for a two-tailed test. $p-value = 2 \cdot P (Z \geq ∣ - 2.8 ∣) = 2 \cdot P (Z \geq 2.8) = 0.00511$ . Why this step? The bell is symmetric — the left tail beyond $- 2.8$ has exactly the same area as the right tail beyond $+ 2.8$ . So $∣ z ∣$ , not $z$ , drives the two-tailed p-value.
Decide. Same as Ex 1: reject $H_{0}$ .

Verify: Ex 1 and Ex 2 give the identical p-value ( $0.00511$ ). That is the whole point of two-tailed: direction is irrelevant, only distance from centre matters ✓.

Cell C — One-tailed, right side

Worked example Ex 3 · Is the new model actually better?

Old model: 82% accuracy. New model: sample accuracy $\overset{π}{^} = 0.87$ on $n = 200$ samples. $H_{0} : π = π_{0} = 0.82$ , $H_{1} : π > 0.82$ (we only care if it improved). Here $π_{0} = 0.82$ is the null proportion — the accuracy $H_{0}$ claims the model really has.

Forecast: improvement is positive, one tail only, so we do not double. Will it clear $0.05$ ?

Standard error under $H_{0}$ . Recall from the vocabulary block that $SE$ (standard error) is the wobble of the estimate: $SE = \frac{π _{0} ( 1 - π _{0} )}{n} = \frac{0.82 \cdot 0.18}{200} = \frac{0.1476}{200} = 0.000738 = 0.027166$ . Why this step? When we test a proportion, its own std-dev shrinks like $1/ n$ . This $SE$ is the width of the bell of possible accuracies.
Standardize. $z = \frac{π ^ - π _{0}}{SE} = \frac{0.87 - 0.82}{0.027166} = \frac{0.05}{0.027166} = 1.8405$ . Why this step? Converts "5 percentage points better" into $SE$ units.
One tail only. $p-value = P (Z \geq 1.8405) = 0.03285$ . Why this step? $H_{1}$ is one-directional — a worse model would never make us reject, so its tail doesn't count.
Decide. $0.0329 < 0.05 \Rightarrow$ reject $H_{0}$ . The gain is significant.

Verify: if we had wrongly run this two-tailed we'd get $2 (0.0329) = 0.066 > 0.05$ and fail to reject — so choosing the right tail-count genuinely changes the decision here ✓.

Cell D — One-tailed, effect in the WRONG direction

Worked example Ex 4 · New model is actually worse

Same one-tailed test $H_{1} : π > 0.82$ with $π_{0} = 0.82$ , but the new model scores $\overset{π}{^} = 0.78$ on $n = 200$ . The standard error $SE$ is unchanged from Ex 3 ( $0.027166$ ) because it depends only on $π_{0}$ and $n$ .

Forecast: the model got worse, but our test only "listens" for improvement. What can the p-value even be? Guess whether we can reject.

Standardize. $z = \frac{π ^ - π _{0}}{SE} = \frac{0.78 - 0.82}{0.027166} = \frac{- 0.04}{0.027166} = - 1.4724$ . Why this step? Negative $z$ = observation is below the null — the opposite side from what $H_{1}$ predicts.
Right-tail p-value. $p-value = P (Z \geq - 1.4724) = 0.9295$ . Why this step? A right-tailed test measures "how much of the bell is above my $z$ ." Since $z$ is on the left, almost the whole bell is above it → huge p-value.
Decide. $0.9295 > 0.05 \Rightarrow$ fail to reject $H_{0}$ , correctly. A worse model can never be evidence for "better."

Verify: $P (Z \geq - 1.4724) = 1 - P (Z \geq 1.4724) = 1 - 0.0705 = 0.9295$ ✓. A p-value above $0.5$ is the tell-tale sign your effect went the wrong way for a one-tailed test.

Cell E — The exact boundary: p-value $= α$

Worked example Ex 5 · Landing precisely on the line

A test yields $z = 1.959964$ in a two-tailed test with $α = 0.05$ . Reject or not?

Forecast: $z \approx 1.96$ is the famous "95% critical value." Does exactly-on-the-line reject?

Compute the p-value. $p-value = 2 \cdot P (Z \geq 1.959964) = 2 (0.025) = 0.05$ . Why this step? $1.959964$ is defined so that each tail holds exactly $2.5%$ ; doubled that is exactly $5%$ .
Apply the rule literally. The decision rule is "reject if the p-value $< α$ " — strict inequality. Here the p-value $= 0.05$ is not less than $0.05$ . Why this step? The convention is $<$ , not $\leq$ . On the exact boundary we fail to reject.
Decide. p-value $= 0.05 \neq < 0.05 \Rightarrow$ fail to reject $H_{0}$ (by a hair).

Verify: the critical value that makes the p-value $= α = 0.05$ two-tailed is $z^{*} = 1.959964$ ; our $z$ equals it exactly, so the p-value $= α$ exactly ✓. This edge is why papers report the p-value to several digits.

Cell F — Degenerate input: observation equals the null

Cell G — Limiting case: huge sample size

Worked example Ex 7 · A tiny effect becomes "significant"

Same 82%→82.5% question, but now $n = 100000$ . So $\overset{π}{^} = 0.825$ , $π_{0} = 0.82$ . One-tailed $H_{1} : π > 0.82$ .

Forecast: the effect (half a percentage point) is trivial. But $n$ is enormous. What happens to $z$ ?

Standard error shrinks. $SE = \frac{π _{0} ( 1 - π _{0} )}{n} = \frac{0.82 \cdot 0.18}{100000} = 0.000001476 = 0.0012149$ . Why this step? Because $SE \propto 1/ n$ , multiplying $n$ by 500 (from 200) shrinks $SE$ by $500 \approx 22 \times$ .
Standardize. $z = \frac{π ^ - π _{0}}{SE} = \frac{0.825 - 0.82}{0.0012149} = \frac{0.005}{0.0012149} = 4.1156$ . Why this step? Same tiny numerator (0.005) over a much smaller denominator → a huge $z$ .
p-value. $p-value = P (Z \geq 4.1156) = 0.00001929$ . Why this step? $z$ past 4 is deep in the tail; the area is minuscule.
Decide. p-value $≪ 0.05 \Rightarrow$ reject $H_{0}$ . "Statistically significant"… yet the effect is a mere $0.5%$ .

Verify: compare to Ex 3 (same effect size $0.05$ in raw terms, $n = 200$ , $z = 1.84$ ). Here the effect is $10 \times$ smaller yet $z$ is $2.2 \times$ bigger, purely from $n$ ✓.

Cell H — Real-world word problem (you build the hypotheses)

Worked example Ex 8 · Does a new checkout page reduce cart abandonment?

Historically 30% of users abandon their cart. After a redesign, in $n = 400$ sessions, $X = 108$ abandon (here $X$ is the count of abandonments). Did the redesign lower abandonment?

Forecast: you must translate English → $H_{0}, H_{1}$ . "Lower" is a direction → one-tailed. Which tail?

Frame it. $H_{0} : π = π_{0} = 0.30$ (redesign did nothing). $H_{1} : π < 0.30$ (redesign reduced abandonment). One-tailed, left. Why this step? The business question only cares about improvement (a drop). We're listening on the left tail.
Observed rate. $\overset{π}{^} = X / n = 108/400 = 0.27$ . Why this step? Convert the raw count $X$ to the proportion the test compares against $π_{0}$ .
Standard error. $SE = \frac{π _{0} ( 1 - π _{0} )}{n} = \frac{0.30 \cdot 0.70}{400} = \frac{0.21}{400} = 0.000525 = 0.022913$ .
Standardize. $z = \frac{π ^ - π _{0}}{SE} = \frac{0.27 - 0.30}{0.022913} = \frac{- 0.03}{0.022913} = - 1.30931$ . Why this step? Negative $z$ is good news here — abandonment dropped, which is the direction $H_{1}$ predicts.
Left-tail p-value. $p-value = P (Z \leq - 1.30931) = 0.09524$ . Why this step? For $H_{1} : π < π_{0}$ we measure the area below our $z$ — the "even bigger drop" region.
Decide. $0.0952 > 0.05 \Rightarrow$ fail to reject $H_{0}$ . Encouraging drop, but not yet convincing at $α = 0.05$ .

Verify: $P (Z \leq - 1.30931) = P (Z \geq 1.30931) = 0.09524$ by symmetry ✓. Practical note: this is a "don't stop the experiment yet, collect more data" result, not "the redesign failed."

Cell I — Exam twist: run the machine backwards

Worked example Ex 9 · Given a target p-value, find the required

$z$ (and count) An exam asks: "For a two-tailed test at $α = 0.01$ , what critical $∣ z ∣$ must the observed statistic exceed? For the coin ( $n = 100$ , $σ = 5$ , $μ = 50$ ), how many heads does that correspond to?"

Forecast: we invert the p-value formula. Higher standard (0.01 vs 0.05) means we need a bigger $z$ .

Split the significance. Two-tailed $α = 0.01$ puts $0.005$ in each tail. Why this step? The critical value is where the right tail area equals $α /2$ .
Invert the normal. We need the $z^{*}$ whose right-tail area is exactly $0.005$ , i.e. $P (Z \geq z^{*}) = 0.005$ . Reading the standard normal backwards — its quantile function, which takes a probability and returns the $z$ that produces it — gives $z^{*} = 2.575829$ . Why this step? The forward direction ( $z \to$ area) is what we've done all page. Here we know the area and want the $z$ ; that is exactly what the quantile (inverse-CDF) function returns.
Convert to a head count. Reverse the standardizing formula: $z = (X - μ) / σ \Rightarrow X = μ + z σ$ . So $X = μ + z^{*} σ = 50 + 2.575829 (5) = 50 + 12.8791 = 62.8791$ . Why this step? We had turned counts into $z$ ; now we turn a target $z$ back into a count so the answer is in the units the question asked (heads).
Round to a whole count. Heads come in whole numbers, so we round the boundary $62.8791$ up to $63$ (you need to strictly exceed the critical value to reject). By symmetry the left boundary rounds to $37$ . Why this step? A count of $62$ would fall short of $62.8791$ ; only $63$ or more clears it.
State the answer. Any observed $∣ z ∣ > 2.5758$ — equivalently $X \geq 63$ heads or $X \leq 37$ heads — rejects $H_{0}$ at $α = 0.01$ .

Verify: $2 \cdot P (Z \geq 2.575829) = 0.01$ exactly ✓, and $50 + 2.575829 \cdot 5 = 62.8791$ ✓. Compare thresholds: $63$ heads clears the strict $α = 0.01$ bar, while the $64$ of Ex 1 clears the easier $α = 0.05$ bar too — stricter $α$ , higher hurdle.

Wrapping the matrix

Every cell of the scenario matrix is now a solved example. The single skill underneath all nine:

$z = \frac{observed - null}{standard error}, p-value = (tails) \cdot P (Z beyond z)$

The only judgement calls are how many tails (from $H_{1}$ 's wording) and which side (from the sign of the effect). Get those two right and the arithmetic is identical every time.

Recall Quick self-test

Two-tailed $z = - 2.8$ : what is the p-value? ::: $2 \cdot P (Z \geq 2.8) = 0.00511$ — the sign folds into $∣ z ∣$ . One-tailed right test, but the effect went the wrong way ( $z < 0$ ): can you ever reject? ::: No — the right-tail p-value will exceed $0.5$ ; a wrong-direction effect is never evidence for $H_{1}$ . Observation exactly equals the null: what is $z$ and the p-value? ::: $z = 0$ , p-value $= 1$ (two-tailed) — maximally boring. Why does a $0.5%$ effect reject $H_{0}$ at $n = 100000$ but not $n = 200$ ? ::: SE shrinks like $1/ n$ , so $z$ grows with $n$ even when the effect is fixed. What's the difference between $π$ , $π_{0}$ , and $\overset{π}{^}$ ? ::: $π$ is the true unknown proportion; $π_{0}$ is the value $H_{0}$ claims; $\overset{π}{^}$ is what the sample showed. What's the difference between capital $Z$ and lowercase $z$ ? ::: $Z$ is the random variable (the whole bell); $z$ is the single observed number our data produced.

See also 1.3.21-Type-I-and-Type-II-errors (what "reject" costs you when $H_{0}$ was true) and 2.5.7-Statistical-significance-in-experiments (how these tests power real A/B decisions).