Foundations — Bayes' theorem — derivation and applications

Before you can read a single line of the parent note, you must be fluent in the symbols it throws at you. This page builds each one from nothing, in the order they depend on each other. Never do we use a symbol we have not first drawn.

---

0. The sample space $S$ — the whole picture

Picture: a rectangle. Every dot inside it is one thing that could happen (one patient, one bulb, one coin toss). Nothing exists outside the rectangle.

Why the topic needs it: Bayes' theorem is about probabilities, and a probability is always "what fraction of the whole?". Without a fixed whole, "fraction" has no meaning.

---

1. An event $A$ — a slice of the whole

Picture: a coloured blob inside the rectangle. "The patient is sick" ($D$) is one blob; "the test reads positive" ($+$) is another.

Why the topic needs it: the whole point of Bayes is to relate one event (a cause, like disease) to another event (the evidence, like a positive test). Events are the nouns of the whole theory.

---

2. Probability $P(A)$ — how big is the slice?

$0\le P(A)\le 1,P(S)=1.$

Picture: if the blob $A$ covers a quarter of the paper, $P(A)=0.25$. If it covers everything, $P(A)=P(S)=1$. If it covers nothing, $P(A)=0$.

Why the topic needs it: priors, likelihoods and posteriors are all probabilities — all "areas of blobs". This single reading powers the whole page.

---

3. The complement $\bar{A}$ — everything that is NOT $A$

Picture: shade the rectangle outside the blob. Together, $A$ and $\bar{A}$ tile the whole sheet with no gaps and no overlap.

$P(A)+P(\bar{A})=1.$

Why the topic needs it: in the medical example, a positive test comes either from a sick person ($D$) or a healthy one ($\bar{D}$). The parent note uses $P(+\mid \bar{D})$ constantly — that "$\bar{D}$" is this idea.

---

4. Intersection $A\cap B$ — the overlap

Picture: two overlapping blobs; the lens-shaped middle is $A\cap B$.

Why the topic needs it: the entire derivation of Bayes hinges on the fact that $P(A\cap B)$ is one single number that can be reached two different ways. That shared overlap is the pivot of Step 1 in the parent note.

---

5. Conditional probability $P(A\mid B)$ — zoom into a slice

This is the star of the show. The vertical bar $\mid$ is read "given".

$P(A\mid B)=\frac{P(A\cap B)}{P(B)},P(B)\ne 0.$

WHAT this formula does: the numerator $P(A\cap B)$ is the overlap; the denominator $P(B)$ is the new whole (the blob $B$ becomes your entire universe).

WHY divide by $P(B)$: because you've zoomed in. $B$ is now the total area, so to keep probabilities summing to $1$ inside this new world, you must re-scale by dividing by $B$'s area.

WHAT IT LOOKS LIKE: shrink the rectangle down to just the blob $B$. The overlap $A\cap B$ was a small lens; now measured against the smaller blob $B$ it becomes a bigger fraction. That re-scaling is the division.

Why the topic needs it: Bayes' theorem is made of conditional probabilities — the likelihood $P(B\mid A)$ and the posterior $P(A\mid B)$ are both of this form. Everything else on the parent page is bookkeeping around this one definition.

---

6. The partition — cutting the whole into causes

Picture: slice the rectangle into coloured stripes with no gaps and no overlaps — like a chocolate bar. $\{A,\bar{A}\}$ is the simplest partition (two pieces).

Why the topic needs it: the denominator of full Bayes, $P(B)={\sum }_{i}P(B\mid A_i)P(A_i)$, only works if the causes tile the world completely. This is the Law of Total Probability, and it needs a partition to stand on.

---

7. The summation sign $\sum$ — "add up over all pieces"

Picture: a hand pointing at each stripe of the chocolate bar in turn, dropping each one's value into a running total.

${\sum }_{i=1}^3{x}_i=x_1+x_2+x_3.$

Why the topic needs it: with more than two causes, writing $P(B\mid A_1)P(A_1)+P(B\mid A_2)P(A_2)+\cdots$ is exhausting. $\sum$ compresses it. The parent's full-Bayes denominator is just this sign at work.

---

8. Building $P(B)$ — the Law of Total Probability

Now watch the pieces click together. Evidence $B$ (a blob) is chopped by the partition into pieces, one per cause:

$P(B)=\underset{\text{slice in }{A}_1}{\underbrace{P(B\cap A_1)}}+\underset{\text{slice in }{A}_2}{\underbrace{P(B\cap A_2)}}+\cdots$

WHY: since the $A_i$ tile the whole sheet with no overlap, the blob $B$ is diced into non-overlapping slivers — add the slivers to recover all of $B$.

Then rewrite each joint using Step 5's formula ($P(B\cap A_i)=P(B\mid A_i)P(A_i)$):

$P(B)={\sum }_{i=1}^{n}P(B\mid A_i)P(A_i).$

This is the denominator the parent note "expands". You now understand every symbol in it.

---

9. Prior, likelihood, posterior — three jobs, one bar

The parent gives these names. Now that you have $P$ and $\mid$, they are just three readings of the same bar:

Picture: prior looks at the whole sheet; likelihood zooms into the cause blob; posterior zooms into the evidence blob. Bayes' theorem is the rule that swaps which blob you zoom into. Compare Prior and Posterior Distributions for how this idea generalises.

---

Prerequisite map

---

Equipment checklist

Test yourself. Cover the right side; you are ready for the parent note only if every reveal matches your own words.