Bayesian statistics — prior, likelihood, posterior (intro): Worked examples

This page is the drill-ground for the parent topic. Every worked example is tagged with the cell of a scenario matrix it fills, so that after reading you will have seen every kind of case Bayes' theorem can throw at you: discrete and continuous, rare and common events, strong and weak priors, zero-data and infinite-data limits, and a couple of exam-style twists.

If a symbol appears that you don't recognise, we build it here — no assumed notation.

The scenario matrix

Think of every Bayes problem as living in one cell of this grid. The columns are what kind of unknown you have; the rows are the tricky feature that changes the answer's flavour.

Cell	Feature being tested	Example that hits it
A	Discrete unknown, ordinary numbers	Ex 1 — disease test
B	Rare event (tiny prior swamps a good test)	Ex 2 — very rare disease
C	Sequential updating (posterior becomes next prior)	Ex 3 — two tests in a row
D	Continuous unknown, flat prior	Ex 4 — coin, uniform prior
E	Continuous unknown, informative prior	Ex 5 — coin, Beta(a,b) prior
F	Zero-data / degenerate limit	Ex 6 — flip nothing, flip all-heads
G	Large-data limit (data drowns the prior)	Ex 7 — 700 heads in 1000
H	Real-world word problem with a twist	Ex 8 — spam filter
I	Exam twist: prior odds & Bayes factor	Ex 9 — odds form

Every numeric answer below is machine-checked in the verify block.

Before we start, two words we will use constantly:

Cell A — Discrete unknown, ordinary numbers

Worked example Ex 1 — Disease test, 1% prevalence

Disease affects $1%$ of people. Test is $99%$ sensitive ( $P (+ ∣ sick) = 0.99$ ) and $95%$ specific ( $P (- ∣ healthy) = 0.95$ , so false-positive rate $0.05$ ). You test positive — so here the data is $D = +$ . Find $P (sick ∣ +)$ .

Forecast: Write down a gut guess now. Most people say "about $99%$ ." Hold that thought.

Step 1 — list the two hypotheses and the prior. $θ \in {sick, healthy}$ , $P (sick) = 0.01$ , $P (healthy) = 0.99$ . Why this step? Bayes needs an exhaustive list of hypotheses with prior weights; here just two.

Step 2 — write the likelihood of the data ( $+$ ) under each. $P (+ ∣ sick) = 0.99$ , $P (+ ∣ healthy) = 0.05$ . Why this step? The likelihood is how well each hypothesis explains what we saw — see Conditional Probability.

Step 3 — evidence via the Law of Total Probability. $P (+) = 0.99 (0.01) + 0.05 (0.99) = 0.0099 + 0.0495 = 0.0594$ Why this step? $P (+)$ must average over all ways a $+$ can arise.

Step 4 — posterior. $P (sick ∣ +) = \frac{0.99 \times 0.01}{0.0594} = \frac{0.0099}{0.0594} \approx 0.1667$ Why this step? This is Bayes' theorem itself: dividing "the one path that is both sick and $+$ " by "all paths to $+$ " flips the conditioning from $P (+ ∣ sick)$ (which we knew) to $P (sick ∣ +)$ (which we want).

Verify: Cross-check with the complement: $P (healthy ∣ +) = \frac{0.0495}{0.0594} = 0.8333$ , and $0.1667 + 0.8333 = 1$ . ✓ The two posterior probabilities partition certainty.

Only ~17% — the $1%$ base rate crushes the " $99%$ test" gut guess. There are simply far more healthy people to generate false positives.

Cell B — Rare event (tiny prior swamps a good test)

Worked example Ex 2 — Same test, prevalence dropped to 0.1%

Now the disease affects only $0.1%$ ( $P (sick) = 0.001$ ). Test is $99%$ accurate both ways. You test positive (data $D = +$ ). Find $P (sick ∣ +)$ .

Forecast: Guess before reading. Will it be higher or lower than Ex 1's $17%$ ?

Step 1 — priors. $P (sick) = 0.001$ , $P (healthy) = 0.999$ . Why this step? The prevalence is the prior; making it $10 \times$ smaller than Ex 1 is the only change, so it must be stated first to isolate its effect.

Step 2 — likelihoods. $P (+ ∣ sick) = 0.99$ , $P (+ ∣ healthy) = 0.01$ . Why this step? These describe how each hypothesis explains the $+$ ; " $99%$ accurate both ways" fixes the sensitivity at $0.99$ and the false-positive rate at $0.01$ .

Step 3 — evidence. $P (+) = 0.99 (0.001) + 0.01 (0.999) = 0.00099 + 0.00999 = 0.01098$ Why this step? Again the Law of Total Probability — we need the total chance of a $+$ across both hypotheses before we can normalise.

Step 4 — posterior. $P (sick ∣ +) = \frac{0.00099}{0.01098} \approx 0.0902$ Why this step? Bayes' theorem: the true-positive path divided by all positive paths flips $P (+ ∣ sick)$ into the answer we want, $P (sick ∣ +)$ .

Verify: Sanity — halving-and-more the prevalence pushed the answer down from $17%$ to $9%$ , exactly the "rarer disease $\Rightarrow$ more false positives" trend. ✓

Figure — Bayesian statistics — prior, likelihood, posterior (intro)

Cell C — Sequential updating (posterior becomes the next prior)

Worked example Ex 3 — Two independent tests, both positive

Prevalence $1%$ , each test $99%$ sensitive and $95%$ specific (as in Ex 1). You take the test twice, independently, and both come back positive (data $D = + +$ ). Find $P (sick ∣ + +)$ .

Forecast: After Ex 1 gave $17%$ from one positive, what does a second positive do?

Step 1 — use Ex 1's posterior as the new prior. After the first $+$ , $P (sick) = 0.1667$ . Why this step? The Bayesian update is recursive: yesterday's posterior is today's prior. Order doesn't matter for independent data.

Step 2 — same likelihoods for the second test. $P (+ ∣ sick) = 0.99$ , $P (+ ∣ healthy) = 0.05$ . Why this step? The test hasn't changed and the second result is independent of the first, so its likelihoods are identical to the first test's — nothing about the machine got better or worse.

Step 3 — evidence for the 2nd test. $P (+_{2}) = 0.99 (0.1667) + 0.05 (0.8333) = 0.16500 + 0.04167 = 0.20667$ Why this step? We normalise the second update against the updated prior ( $0.1667$ / $0.8333$ ), not the original $0.01$ — the total probability of a second $+$ must reflect what we now believe.

Step 4 — posterior after both. $P (sick ∣ + +) = \frac{0.99 \times 0.1667}{0.20667} \approx 0.7984$ Why this step? Bayes' theorem once more, feeding in the post-first-test belief; this delivers the belief after both positives.

Verify: Do it in one shot instead — multiply likelihoods for two positives: $\frac{0.9 9 ^{2} ( 0.01 )}{0.9 9 ^{2} ( 0.01 ) + 0.0 5 ^{2} ( 0.99 )} = \frac{0.009801}{0.009801 + 0.0024750} \approx 0.7984$ Same number ✓ — confirming sequential updating $=$ batch updating.

A single positive left us at $17%$ ; a second independent positive vaults us to ~80%. Accumulating evidence, done honestly.

Cell D — Continuous unknown, flat prior

Worked example Ex 4 — Coin bias, 7 heads in 10, uniform prior

Unknown bias $θ \in [0, 1]$ . The data is $D =$ " $7$ heads in $10$ tosses". Prior $P (θ) = 1$ . Find the posterior and its mean.

Forecast: Will the posterior mean be exactly $0.7$ , or pulled somewhere?

Step 1 — likelihood is Binomial. $P (D ∣ θ) = (7 10) θ^{7} (1 - θ)^{3}$ Why this step? Each head contributes a factor $θ$ , each tail a factor $(1 - θ)$ ; the binomial coefficient just counts orderings and is constant in $θ$ .

Step 2 — proportional posterior. $P (θ ∣ D) \propto θ^{7} (1 - θ)^{3} \cdot 1$ Why this step? posterior $\propto$ likelihood $\times$ prior; the flat prior is $1$ and the constant $(7 10)$ is absorbed into normalization.

Step 3 — recognise the family. $θ^{8 - 1} (1 - θ)^{4 - 1} \Rightarrow θ ∣ D \sim Beta (8, 4)$ . Why this step? Beta is conjugate to the Binomial: Beta $(1, 1) + 7$ heads $+ 3$ tails $=$ Beta $(8, 4)$ .

Step 4 — posterior mean. $\frac{a}{a + b} = \frac{8}{12} = 0.6 \overline{6}$ .

Verify: The mean $0.667$ sits between the prior mean $0.5$ and the data fraction $0.7$ — the prior shrinks the estimate slightly toward $0.5$ . ✓

Cell E — Continuous unknown, informative prior

Worked example Ex 5 — Same 7/10 data, but a Beta(2,2) prior

You believe beforehand the coin is roughly fair: prior $= Beta (2, 2)$ (a gentle hump at $0.5$ ). The data is again $D =$ " $7$ heads in $10$ ". Find the posterior and its mean.

Forecast: Compared with Ex 4's $0.667$ , will the mean move toward or away from $0.5$ ?

Step 1 — prior as clay. $Beta (2, 2) \propto θ^{1} (1 - θ)^{1}$ . Step 2 — multiply by likelihood. $P (θ ∣ D) \propto θ^{7} (1 - θ)^{3} \cdot θ^{1} (1 - θ)^{1} = θ^{8} (1 - θ)^{4}$ Why this step? Conjugacy again: just add the counts. Beta $(2, 2)$ with $7$ heads, $3$ tails $\to$ Beta $(2 + 7, 2 + 3)$ . Step 3 — posterior. $Beta (9, 5)$ . Step 4 — mean. $\frac{9}{9 + 5} = \frac{9}{14} \approx 0.6429$ .

Verify: $0.6429 < 0.6667$ : the stronger "fair" prior pulled the estimate further toward $0.5$ than the flat prior did. ✓ In the figure below the plum posterior sits to the left of the orange one — you can literally see the extra pull.

Cell F — Zero-data and degenerate limits

Worked example Ex 6 — What if you observe (a) nothing, (b) all heads?

Coin with flat prior Beta(1,1). (a) You flip zero times (data $D = \emptyset$ ). What is the posterior? (b) You flip 3 times, get 3 heads (data $D =$ " $3$ heads"). What is the posterior mean, and is $θ = 1$ certain?

Forecast: For (b), does 3-for-3 prove the coin always lands heads?

Part (a) — zero data. Step: With no data, likelihood is constant ( $1$ ) for every $θ$ , so posterior $\propto$ prior. Why? No data $\Rightarrow$ nothing to update $\Rightarrow$ posterior $=$ prior $=$ Beta $(1, 1)$ , mean $0.5$ . Verify (a): posterior $=$ prior exactly. ✓ Degenerate case handled: Bayes never invents information.

Part (b) — all heads. Step 1: likelihood $\propto θ^{3} (1 - θ)^{0} = θ^{3}$ . Step 2: posterior $\propto θ^{3} \Rightarrow Beta (4, 1)$ . Step 3: mean $= \frac{4}{4 + 1} = 0.8$ . Why not $1$ ? A proper prior (Beta $(1, 1)$ ) keeps a sliver of belief that $θ < 1$ ; the posterior peaks at the edge but its mean stays at $0.8$ , not $1$ . Bayes refuses to be certain from finite data.

Verify (b): mean $= 0.8$ and $P (θ = 1 ∣ D) = 0$ since a continuous density puts zero mass on a single point. ✓ Compare with Maximum Likelihood Estimation, which would return $\hat{θ} = 1$ — a cautionary contrast between the two philosophies.

Cell G — Large-data limit (data drowns the prior)

Worked example Ex 7 — 700 heads in 1000, Beta(2,2) prior

Strongly-fair prior Beta $(2, 2)$ , but now a mountain of data: $D =$ "700 heads in 1000 tosses". Posterior mean?

Forecast: In Ex 5 the same prior pulled $0.7$ down to $0.643$ . With $1000$ tosses, how much pull survives?

Step 1 — add counts. Beta $(2 + 700, 2 + 300) = Beta (702, 302)$ . Why this step? Conjugacy makes updating pure bookkeeping: add heads to $a$ , tails to $b$ . Here the two prior "pseudo-counts" are tiny next to $700$ and $300$ real counts — a preview of why the prior barely moves the needle.

Step 2 — mean. $\frac{702}{1004} \approx 0.6992$ . Why this step? The posterior mean $\frac{a}{a + b}$ is our single-number summary; computing it lets us measure how much of the prior's pull survived the flood of data.

Verify: $0.6992$ is a hair under the raw data fraction $0.700$ — the prior's influence has almost vanished. ✓ As data grows, the Bayesian estimate converges to the MLE / frequentist answer. The prior only matters when data is scarce.

Cell H — Real-world word problem (twist)

Worked example Ex 8 — Spam filter with two clue words

A naive-Bayes spam filter. Prior: $30%$ of email is spam. In spam, the word "free" appears with prob $0.6$ ; in ham, prob $0.1$ . In spam, "meeting" appears with prob $0.05$ ; in ham, prob $0.4$ . An email contains both "free" and "meeting" — that pair is our data $D$ . Is it spam?

Forecast: One word screams spam, the other screams ham. Which wins?

Step 1 — assume the two words are conditionally independent given class (the "naive" bit). Why this step? It lets us multiply per-word likelihoods instead of estimating impossible joint tables. Step 2 — likelihood of the pair. $P (words ∣ spam) = 0.6 \times 0.05 = 0.03, P (words ∣ ham) = 0.1 \times 0.4 = 0.04$ Step 3 — weight by priors. $spam score = 0.03 \times 0.30 = 0.009, ham score = 0.04 \times 0.70 = 0.028$ Step 4 — normalise. $P (spam ∣ words) = \frac{0.009}{0.009 + 0.028} = \frac{0.009}{0.037} \approx 0.2432$

Verify: $P (ham ∣ words) = \frac{0.028}{0.037} \approx 0.7568$ and the two sum to $1$ . ✓ Despite "free", the email is ~76% ham — "meeting" plus the ham-heavy prior win.

Cell I — Exam twist: odds form & Bayes factor

Worked example Ex 9 — Redo Ex 1 in odds form

Prevalence $1%$ , sensitivity $0.99$ , false-positive $0.05$ , observed $D = +$ . Find $P (sick ∣ +)$ via odds.

Forecast: Should land on the same $17%$ as Ex 1 — a consistency check on the method.

Step 1 — prior odds. $\frac{P ( sick )}{P ( healthy )} = \frac{0.01}{0.99} = \frac{1}{99}$ . Step 2 — Bayes factor. $\frac{P ( + ∣ sick )}{P ( + ∣ healthy )} = \frac{0.99}{0.05} = 19.8$ . Step 3 — posterior odds. $19.8 \times \frac{1}{99} = 0.2$ . Step 4 — odds back to probability. $p = \frac{odds}{1 + odds} = \frac{0.2}{1.2} \approx 0.1667$ . Why this step? Odds and probability are two dials for the same belief; the identity $p = \frac{o}{1 + o}$ is just the [!definition] odds relation $o = \frac{p}{1 - p}$ solved for $p$ , so we convert back to report a familiar percentage.

Verify: $0.1667$ matches Ex 1 exactly. ✓ Same physics, one line of arithmetic. This is the form to reach for under exam time-pressure.

Recall Which cell am I in? (self-test — reveal after guessing)

"Rare disease, one positive test" ::: Cell A/B — discrete, base-rate dominated. "I already updated once and got more data" ::: Cell C — posterior becomes the new prior. "Coin bias with no strong opinion beforehand" ::: Cell D — continuous, flat Beta(1,1). "Coin bias but I truly believe it's near fair" ::: Cell E — informative Beta prior. "I have thousands of observations" ::: Cell G — data drowns the prior, converges to MLE. "I only need to compare two hypotheses fast" ::: Cell I — odds form, evidence cancels.