Bayes' theorem — derivation and applications: Question bank

Before the traps, let us earn every symbol used below so nothing appears unexplained.

Now the core object under fire — the flip, with the partition already defined above: $P (A ∣ B) = \frac{P ( B ∣ A ) P ( A )}{P ( B )}, P (B) = \sum_{i = 1}^{n} P (B ∣ A_{i}) P (A_{i}) .$ The sum runs over the partition $A_{1}, \dots, A_{n}$ : it adds the probability of the evidence $B$ arriving through each separate cause. Every item below pokes at one symbol in that line — the prior $P (A)$ , the likelihood $P (B ∣ A)$ , the evidence $P (B)$ , or the direction of the bar $∣$ .

True or false — justify

A test that is

99%

sensitive (i.e.

P (+ ∣ D) = 0.99

) means a positive result implies a

99%

chance of disease.

False. Sensitivity is

P (+ ∣ D)

, the test's behaviour given the truth; the patient wants the flipped

P (D ∣ +)

, which also depends on the prior

P (D)

— a rare disease drags it far below

99%

. In the area picture, the left strip is thin, so most shaded area sits on the right.

Saying a test is "

99%

accurate" is the same as saying its sensitivity is

99%

False. "Accuracy" is the overall correct-rate, blending sensitivity and specificity weighted by prevalence; a test can be

99%

sensitive yet poorly specific, so its accuracy differs. The words are not interchangeable — always ask which quality number is meant.

P (A ∣ B) = P (B ∣ A)

whenever

A

and

B

are equally common.

True (as a special case). They differ by the factor

P (A) / P (B)

, so when

P (A) = P (B)

the two conditionals coincide — but this is a coincidence of base rates, not a general law.

P (A ∣ B) > P (A)

, then seeing

B

made

A

more believable.

True. The posterior rose above the prior, so evidence

B

supported

A

; equivalently

P (B ∣ A) > P (B)

, meaning

B

is "more expected" when

A

holds.

For independent events, Bayes' theorem still holds but tells you nothing new.

True. If

A, B

are independent then

P (A ∣ B) = P (A)

: the posterior equals the prior, so evidence

B

leaves belief in

A

unchanged — the two vertical strips shade to the same height.

The denominator

P (B)

can be omitted because it cancels out.

False.

P (B)

is what normalises posteriors to sum to

1

across all causes; drop it and you get relative weights, not probabilities. It only "cancels" when you compare two hypotheses as a ratio.

A posterior of

16.7%

from a positive test means the test is useless.

False. It still lifted belief from the

1%

prior to

16.7%

— a roughly

17

-fold jump. "Useless" would be the posterior staying at

1%

If the prior is

0

, no amount of evidence can produce a nonzero posterior.

True.

P (A) = 0

makes the numerator

P (B ∣ A) P (A) = 0

; Bayes can update beliefs but cannot resurrect a hypothesis you assigned literally impossible.

Swapping the roles of "cause" and "evidence" in Bayes gives an equally valid theorem.

True. Bayes is symmetric in construction — you may compute

P (B ∣ A)

from

P (A ∣ B)

just as easily; "cause" and "evidence" are our interpretive labels, not fixed algebraic roles.

Spot the error

P (D) = 0.01

and

P (+ ∣ D) = 0.99

, so

P (D ∣ +) = 0.99 \times 0.01 = 0.0099

Error: that product is the joint

P (D \cap +)

— the area of the tall sliver inside the left strip — not the posterior. You still must divide by the total shaded area

P (+) = 0.0594

, giving

\approx 0.167

"The disease is rare, so I'll ignore the healthy group when computing

P (+)

Error: the healthy group (the right strip) is exactly where the false positives live. The short band there,

P (+ ∣ \overset{ˉ}{D}) P (\overset{ˉ}{D}) = 0.05 \times 0.99 = 0.0495

, is five times the true-positive sliver

0.0099

— ignoring it discards most of the shaded area.

"Factory Y makes fewer bulbs, so a defective bulb is more likely from X."

Error: conditioning on "defective" reweights by defect rate. Y's

0.05

vs X's

0.02

flips the balance to

P (Y ∣ D) = 0.625 > P (X ∣ D) = 0.375

"My causes are

A_{1}

='sick',

A_{2}

='healthy',

A_{3}

='tired'; sum their likelihoods for

P (B)

Error: the causes must form a partition (mutually exclusive and exhaustive). "Tired" overlaps both "sick" and "healthy", so the slices double-count —

P (B)

will not normalise correctly.

"The court found

P (match ∣ innocent) = 1

in a million, so

P (innocent ∣ match) = 1

in a million."

Error: the prosecutor's fallacy — the bar

∣

was silently flipped. The true posterior depends on the prior number of possible matches (the base rate of the population), often making guilt far from certain.

P (A ∣ B) + P (A ∣ \overset{ˉ}{B}) = 1

, since

B

and its complement

\overset{ˉ}{B}

cover everything."

Error: what sums to

1

P (A ∣ B) + P (\overset{ˉ}{A} ∣ B)

(over the left argument for a fixed right event). Conditioning on different events

B

and

\overset{ˉ}{B}

gives probabilities from two separate shrunken worlds that need not add up.

Why questions

Why does a higher disease prevalence make the same test more trustworthy?

A larger prior

P (D)

widens the left strip, enlarging the true-positive sliver relative to the fixed false-positive band, so the posterior fraction climbs (from

16.7%

1%

89.5%

30%

Why do we write conditional probability "both ways" as the first derivation step?

Both directions share the same joint

P (A \cap B)

— the same overlap region of the two events; writing both lets us eliminate that common term and solve for the flipped conditional.

Why must the causes form a partition before applying the full formula?

So that

B

can occur through exactly one cause with no gaps or overlaps, making

P (B) = \sum_{i} P (B ∣ A_{i}) P (A_{i})

an exact accounting of every path to

B

— the strips tile the whole square with no double-shading.

Why is Bayes called an "updating" rule rather than just an "identity"?

You feed it a prior belief and a fresh datum

B

and it returns a revised belief; yesterday's posterior becomes today's prior, so repeated evidence updates iteratively — the engine behind continuous prior-and-posterior distributions.

Why does the "naive" independence assumption still give useful classifications?

It assumes features are conditionally independent given the class, which is often false; but for ranking classes the shared evidence term cancels, and small dependency errors rarely change which class wins.

Why can two people update to different posteriors from the same evidence?

Because they started from different priors

P (A)

— different strip widths; Bayes multiplies the shared likelihood by each person's own prior, so honest disagreement about starting beliefs propagates to the answer.

Edge cases

What is

P (A ∣ B)

when

P (B) = 0

Undefined. Division by zero — you cannot condition on an event that never occurs, so the definition

P (A \cap B) / P (B)

has no meaning there.

What happens to the posterior when the likelihood

P (B ∣ A) = 0

The numerator vanishes, so

P (A ∣ B) = 0

: if

A

never produces the observed evidence, then observing it rules

A

out entirely.

What if a test is perfectly specific,

P (+ ∣ \overset{ˉ}{D}) = 0

Then no false positives exist — the right strip's shaded band has zero height — so the evidence term reduces to

P (+ ∣ D) P (D)

, the whole numerator, and any positive gives

P (D ∣ +) = 1

: a positive is now conclusive.

What does Bayes give when prior and likelihood point in opposite directions (rare cause, strong evidence)?

A tug-of-war: a tiny prior can be overpowered by a very large likelihood ratio, or can survive it — the outcome is literally their product, which is why both numbers must be quoted, never one alone.

What is the posterior if the evidence is equally likely under every cause,

P (B ∣ A_{i})

all equal?

The evidence is uninformative; the likelihoods cancel in the normalisation and each posterior equals its prior,

P (A_{k} ∣ B) = P (A_{k})

— no update occurs.

What happens as you apply Bayes repeatedly with overwhelming consistent evidence?

The posterior converges toward

1

for the true cause (unless its prior was exactly

0

), because each update multiplies by a likelihood ratio favouring it, drowning out the initial prior.

Recall One-line self-test

If you can answer "given which event?" for every conditional you write, you have already dodged the two deadliest traps here.

Connections

Bayes' theorem — derivation and applications — the parent this bank interrogates.
Conditional Probability — where the direction-of-bar $∣$ confusions originate.
Law of Total Probability — the machinery behind the partition sum $P (B) = \sum_{i} P (B ∣ A_{i}) P (A_{i})$ .
Independent Events — formalises the "no update" edge case.
Prior and Posterior Distributions — the continuous version of iterated updating.
Naive Bayes Classifier — the applied "why it still works" question.
Tree Diagrams — an alternative to the area model for the same bookkeeping.