Bayesian statistics — prior, likelihood, posterior (intro): Visual walkthrough

We only need one idea to start: probability is area. Nothing else.

Step 1 — The whole world is a square of area 1

WHAT. Draw a square. Its total area is $1$ . Every possible outcome lives somewhere inside it. A region inside the square is an event — a thing that might happen — and its probability is just how much of the square it covers.

WHY. We need a picture that turns the word "probability" into something you can measure with a ruler. Area does exactly that: area $\geq 0$ always, and the whole square adds to $1$ , matching the two rules every probability obeys.

PICTURE. Two overlapping blobs sit inside the square. The lavender blob is event $A$ ; the coral blob is event $B$ .

Figure — Bayesian statistics — prior, likelihood, posterior (intro)

Nothing here is abstract yet — $P (A)$ is literally paint coverage.

Step 2 — "Given $B$ " means: throw away the rest of the square

WHAT. Suppose someone tells you " $B$ definitely happened." Now the coral blob $B$ is your entire new world. We zoom in so that $B$ fills the frame, and ask: inside this new world, how much is also $A$ ?

WHY. Conditioning is not multiplication or a new axiom — it is re-scaling. When we learn $B$ is true, the parts of the square outside $B$ became impossible, so we cannot count them anymore. We must re-measure $A$ using $B$ as the new "total".

PICTURE. The left frame shows the old square; the right frame is the same $B$ blown up to be the whole world. The overlap $A \cap B$ is the only surviving piece of $A$ .

See Conditional Probability — this fraction is the whole of it, drawn.

Step 3 — The overlap can be measured from either side

WHAT. That same overlap patch $A \cap B$ can be built two ways:

start in the $B$ world, take the $A$ -fraction of it: $P (A ∣ B) P (B)$ ;
start in the $A$ world, take the $B$ -fraction of it: $P (B ∣ A) P (A)$ .

WHY. The overlap is one physical patch of paint. It does not know which blob we "entered first". So the two recipes must give the same area. This symmetry is the seed of the whole theorem.

PICTURE. The identical purple overlap is reached by two arrows — one slicing $B$ first, one slicing $A$ first — landing on the same square.

Step 4 — Divide once and Bayes falls out

WHAT. Take the two outer pieces of Step 3 and divide both by $P (B)$ : $P (A ∣ B) P (B) = P (B ∣ A) P (A) ⟹ P (A ∣ B) = \frac{P ( B ∣ A ) P ( A )}{P ( B )} .$

WHY. We want $P (A ∣ B)$ — the thing we can't see directly (e.g. "am I sick, given the test?"). We have $P (B ∣ A)$ — the thing labs measure (e.g. "does the test fire, given sickness?"). Dividing by $P (B)$ flips the conditioning bar and isolates what we want.

PICTURE. An arrow labelled "flip the bar" turns $P (B ∣ A)$ (easy to measure) into $P (A ∣ B)$ (what we care about), with the $P (A) / P (B)$ correction riding along.

Step 5 — Rename the blobs for statistics

WHAT. Let $A = θ$ (the hypothesis / parameter — the coin's bias, the disease) and $B = D$ (the data we actually observed). Same square, new labels.

WHY. In statistics we don't see $θ$ ; we see $D$ . So we want $P (θ ∣ D)$ from the ingredients we can get.

PICTURE. The identical two-blob square, now with the lavender blob labelled $θ$ and the coral blob labelled $D$ , and each area tagged with its statistics name.

Step 6 — Where does $P (D)$ come from? Chop the data blob

WHAT. The denominator $P (D)$ is the total area of the coral blob. Split the data blob into the piece inside $θ$ and the piece inside "not- $θ$ " ( $\overset{ˉ}{θ}$ ), then add: $P (D) = data via θ P (D ∣ θ) P (θ) + data via not- θ P (D ∣ \overset{ˉ}{θ}) P (\overset{ˉ}{θ}) .$

WHY. The data can arrive through any hypothesis. To get the true size of $D$ we must sweep over all the ways it could happen — that is the Law of Total Probability. With many hypotheses it becomes a sum $\sum_{θ}$ (or an integral $\int$ for continuous $θ$ ).

PICTURE. The coral $D$ blob is sliced by the boundary of the $θ$ blob into two shaded strips; their areas add to $P (D)$ .

Step 7 — Degenerate cases (never let the reader hit a wall)

WHAT. Check every way the picture can break.

WHY. A formula you trust must survive its extremes, not just the friendly middle.

PICTURE. Four mini-squares: (a) no overlap, (b) $θ$ swallows $D$ , (c) $P (θ) = 0$ , (d) $P (D) = 0$ .

The one-picture summary

Everything at once: two blobs in a unit square. The overlap is the numerator $P (D ∣ θ) P (θ)$ ; the whole coral blob is the denominator $P (D)$ ; their ratio — overlap over data-world — is the posterior $P (θ ∣ D)$ . Bayes is just "the overlap, measured as a slice of the data world."

Recall Feynman retelling — the whole walkthrough in plain words

Picture the world as a square of area $1$ : everything that could happen fits inside. Two paint blobs overlap — "my guess is right" (lavender) and "here's the data I saw" (coral). To update, I forget everything outside the coral blob, because the data really happened — coral is my new world. Now I ask: how much of this coral world is also lavender? That fraction is my new belief, the posterior. The clever trick is that the overlap patch can be measured two ways — starting from either blob — and setting those two equal, then dividing by the coral area, flips the easy lab number ("test fires when you're sick") into the number I actually want ("I'm sick because the test fired"). The bottom, the whole coral area, is just all the ways the data could have shown up, added together. And the warnings: no overlap means the guess is dead; a guess I'd given zero chance to stays dead no matter what I see; and I can never condition on data that was impossible to begin with.

Recall Quick self-test

Why do we divide by $P (D)$ and not $P (θ)$ ? ::: Because "given $D$ " makes $D$ our new world; we re-scale by the size of the world we conditioned on, which is $D$ . In the picture, which region is the numerator of Bayes' theorem? ::: The overlap $A \cap B$ , i.e. $P (D ∣ θ) P (θ)$ . If $P (θ) = 0$ , what is the posterior for any data? ::: Zero — a ruled-out prior can never be revived. What breaks if $P (D) = 0$ ? ::: Division by zero; you cannot condition on impossible data.

Related builds: Binomial Distribution and Beta Distribution give the shapes for the coin example; Maximum Likelihood Estimation keeps only the likelihood (flat prior); Naive Bayes Classifier runs this square millions of times for spam filters.