Foundations — Bayes' theorem and applications

Before you touch the parent note, you must be fluent in the symbols it throws at you without warning. This page builds each one from nothing, in the order they depend on each other, and pins every one to a picture.

0. The picture everything lives in

Every probability statement in this whole chapter is a statement about one rectangle. The rectangle is the set of everything that could possibly happen — one roll of the dice, one random person, one email. We call this rectangle the sample space.

Why we need it: every number in Bayes is a fraction of this rectangle. Probability = area. Once you believe "probability is just how much of the rectangle a shape covers", the rest is geometry.

1. Event — the symbol $A$, $B$, $D$, $T$, $S$, $F$

The parent note writes letters like $A$, $B$, $D$ (disease), $T$ (test positive) and never says what kind of thing a letter is. Here it is:

• $A$ is a blob. $P(A)$ is the area of that blob.
• If the whole rectangle has area $1$, and blob $A$ covers a quarter of it, then $P(A)=0.25$.

2. $P(\cdot )$ — the "measure the area" symbol

Why the topic needs it: priors like $P(D)=0.01$ and posteriors like $P(D|T)\approx 0.161$ are all just areas. "The disease is rare" literally means "the disease blob is a thin sliver."

3. $\neg A$ — "not $A$", the complement

The parent uses $P(\neg A)$ and $P(\neg D)$ constantly (in the denominator especially) but never draws it.

Because the whole rectangle has area $1$ and $A$ plus its outside fill it completely: $P(A)+P(\neg A)=1⟹P(\neg A)=1-P(A)$

Why the topic needs it: in the medical example $P(D)=0.01$, so $P(\neg D)=0.99$. That huge healthy region is why false positives swamp true positives — you cannot see that without drawing $\neg A$.

4. $A\cap B$ — the overlap ("$A$ and $B$")

Two blobs can overlap. The parent's derivation lives entirely in that overlap.

Why the topic needs it: the derivation of Bayes starts from $P(A\cap B)$ and computes it two different ways. That double-counting of the overlap is the whole trick.

5. $P(A\mid B)$ — conditional probability, the star of the show

This is the single most important — and most misread — symbol on the parent page.

What the picture does: conditioning on $B$ means throwing away the rest of the rectangle and treating blob $B$ as your new rectangle. Then $P(A\mid B)$ is just: how much of this new rectangle is the overlap? That is why we divide by $P(B)$ — we are re-measuring areas as fractions of $B$ instead of fractions of the original whole.

Why this tool and not plain $P(A\cap B)$? Because the question changes. "How likely are disease AND positive test together?" (a $P(A\cap B)$ question) is different from "given a positive test, how likely is disease?" (a $P(A\mid B)$ question). Conditioning is the tool that answers the second, which is the one a doctor actually cares about.

6. Multiplication into the joint — why $P(A\cap B)=P(B\mid A)P(A)$

Take the conditional definition and multiply both sides by the denominator: $P(A\mid B)=\frac{P(A\cap B)}{P(B)}⟹P(A\cap B)=P(A\mid B)\cdot P(B)$ and symmetrically $P(A\cap B)=P(B\mid A)\cdot P(A).$

What we just did: we found two recipes for the exact same lens. Why: setting the two recipes equal is literally Bayes' theorem — the parent's whole derivation is "compute the overlap two ways, then solve."

7. Mutually exclusive & exhaustive — why the denominator splits in two

The parent's most-used form expands $P(B)$ into two pieces. That expansion needs two words.

When events are both, blob $B$ gets split cleanly into "the part of $B$ inside $A$" plus "the part of $B$ inside $\neg A$", with nothing double-counted and nothing missing: $P(B)=\underset{\text{part of }B\text{ in }A}{\underbrace{P(B\mid A)P(A)}}+\underset{\text{part of }B\text{ in }\neg A}{\underbrace{P(B\mid \neg A)P(\neg A)}}$

This is the law of total probability, and it is exactly why the parent can compute $P(B)$ (the evidence) even when it is never handed to you directly.

8. $\sum$, $\prod$, subscripts $A_i$, and the vector $\mathbf{x}$

The extended Bayes formula and the Naive Bayes example use compact notation the parent never unpacks.

Why the topic needs them: with two hypotheses you write the sum by hand ($A$ and $\neg A$). With $n$ hypotheses or $n$ features you would run out of ink — $\sum$ and $\prod$ are just "keep going" instructions.

The prerequisite map

Equipment checklist

Cover the right side and test yourself. If any line surprises you, re-read its section above before opening the parent note.