Bayes' theorem — derivation and applications

WHY do we need it?

Often the probability we can measure is the "wrong way round".

A medical test gives us $P(\text{positive} \mid \text{disease})$ — how the test behaves when someone is sick.
But the patient wants $P(\text{disease} \mid \text{positive})$ — given my positive result, am I actually sick?

These two are not equal. Bayes' theorem is the bridge between them.

WHAT are the ingredients?

The definition of conditional probability is: $P(A\mid B) = \frac{P(A\cap B)}{P(B)}, \qquad P(B)\ne 0.$

If causes $A_1,\dots,A_n$ are mutually exclusive and exhaustive (a partition of the sample space), then any evidence $B$ can only happen through one of them: $P(B) = \sum_{i=1}^{n} P(B\mid A_i)\,P(A_i).$

Figure — Bayes' theorem — derivation and applications

Worked Example 1 — The classic false-positive shock

A disease affects $1\%$ of a population. A test is $99\%$ sensitive ( $P(+\mid D)=0.99$ ) and $95\%$ specific ( $P(-\mid \bar D)=0.95$ , so $P(+\mid\bar D)=0.05$ ). You test positive. What is $P(D\mid +)$ ?

Step 1 — list priors and likelihoods. $P(D)=0.01,\; P(\bar D)=0.99,\; P(+\mid D)=0.99,\; P(+\mid \bar D)=0.05.$ Why? We separate what we believe before (prior) from how the test behaves (likelihood).

Step 2 — total probability of a positive. $P(+) = (0.99)(0.01) + (0.05)(0.99) = 0.0099 + 0.0495 = 0.0594.$ Why? A positive can come from a true sick person OR a healthy false positive.

Step 3 — apply Bayes. $P(D\mid +) = \frac{0.0099}{0.0594} \approx 0.1667 = 16.7\%.$ Why the surprise? Because the disease is rare, the huge healthy group produces many false positives that swamp the few true positives.

Worked Example 2 — Two factories (partition)

Factory X makes $60\%$ of bulbs with $2\%$ defect rate; Factory Y makes $40\%$ with $5\%$ defect rate. A bulb is defective. From which factory did it likely come?

Step 1 — priors: $P(X)=0.6,\ P(Y)=0.4$ . Likelihoods: $P(D\mid X)=0.02,\ P(D\mid Y)=0.05$ .

Step 2 — evidence: $P(D)=(0.02)(0.6)+(0.05)(0.4)=0.012+0.020=0.032.$ Why? Sum over both factories (they partition all bulbs).

Step 3 — posteriors: $P(X\mid D)=\frac{0.012}{0.032}=0.375,\qquad P(Y\mid D)=\frac{0.020}{0.032}=0.625.$ Why? Even though X makes more bulbs, Y's higher defect rate makes it the more likely source of a defective one.

Forecast-then-Verify

Common Mistakes

Recall Feynman: explain to a 12-year-old

Imagine a box that flashes "DANGER!" whenever a monster is near — and it's really good, flashing $99$ out of $100$ times a monster comes. But monsters are super rare. So most of the time it flashes, it's actually a squirrel that tricked it. Before you panic, you must think: "How many monsters are there in the first place?" Bayes' theorem mixes how good the alarm is with how rare monsters really are to tell you the true chance. New clue → update your guess.

Active Recall Flashcards

State the definition of conditional probability

P(A\mid B)

P(A\mid B)=\dfrac{P(A\cap B)}{P(B)}

, for

P(B)\neq 0

Write Bayes' theorem in its simplest form.

P(A\mid B)=\dfrac{P(B\mid A)\,P(A)}{P(B)}

What is the "prior" in Bayes' theorem?

P(A)

— belief in the cause before seeing the evidence.

What is the "likelihood"?

P(B\mid A)

— probability of the evidence given the cause is true.

How do you expand

P(B)

for a partition

A_1,\dots,A_n

P(B)=\sum_i P(B\mid A_i)P(A_i)

(Law of Total Probability).

Why can a 99%-accurate test still give a low

P(\text{disease}\mid +)

Because a low prior (rare disease) means false positives from the large healthy group dominate.

Name the fallacy of swapping

P(A\mid B)

and

P(B\mid A)

The prosecutor's fallacy / transposed conditional.

In Example 2, why is the defective bulb more likely from the smaller factory Y?

Y's higher defect rate (0.05) outweighs X's larger share, once we condition on "defective".

Connections

Conditional Probability — the definition Bayes is built from.
Law of Total Probability — supplies the denominator $P(B)$ .
Independent Events — special case where $P(A\mid B)=P(A)$ .
Naive Bayes Classifier — machine-learning application.
Prior and Posterior Distributions — the continuous generalisation.
Tree Diagrams — visual bookkeeping for the joint probabilities.

Concept Map

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Bayes' theorem ka core idea simple hai: conditional probability ko ulta karna. Bahut baar humein pata hota hai $P(\text{evidence}\mid \text{cause})$ — jaise test positive aane ka chance jab bimari hai. Par patient ko chahiye $P(\text{cause}\mid \text{evidence})$ — positive aaya, ab sach mein bimari hai kya? Ye dono barabar nahi hote, aur Bayes theorem in dono ke beech ka pul hai.

Formula banta hai bahut seedhe se: $P(A\cap B)$ ko dono taraf se likho, phir barabar karke divide karo. Result: posterior $=$ (likelihood $\times$ prior) $\div$ evidence. Yaad rakhne ke liye "PLE over E" bol lo. Denominator $P(B)$ ko hum saare causes pe sum karke nikalte hain — isko Law of Total Probability kehte hain.

Sabse bada lesson base rate ka hai. Ek $99\%$ accurate test bhi galat lag sakta hai agar bimari rare hai. Kyunki healthy log itne zyada hain ki unke false positives, real sick logon ke true positives ko dabaa dete hain. Isliye Example 1 mein answer sirf $16.7\%$ aata hai — chaunkane wala, par sahi. Jab prevalence $30\%$ kar do, wahi test $89.5\%$ trustworthy ho jaata hai. Matlab prior aur evidence dono important hain.

Exam tip: kabhi bhi $P(A\mid B)$ aur $P(B\mid A)$ ko mat mix karo (prosecutor's fallacy), aur $P(B)$ nikalte waqt saare causes include karo warna posterior sum $1$ nahi hoga. Bas prior, likelihood, evidence — teeno neatly likh lo, aur answer khud aa jaata hai.

Bayes' theorem — derivation and applications

WHY do we need it?

WHAT are the ingredients?

HOW: derivation from scratch

Expanding the denominator (Law of Total Probability)

Worked Example 1 — The classic false-positive shock

Worked Example 2 — Two factories (partition)

Forecast-then-Verify

Common Mistakes

Active Recall Flashcards

Connections

Concept Map

Hinglish (regional understanding)

Go deeper — visual, from zero

Test yourself — Statistics & Probability — Intermediate

Connections