Conditional probability: Question bank — AI-ML deep dive

This page is a workout for your intuition, not your calculator. Every item below is a trap the topic sets: a place where a smart reader nods along and gets it slightly wrong. Read the prompt, cover the answer, say your reasoning out loud, then reveal.

Everything here builds on the parent note Conditional probability and its neighbours: Bayes-Theorem, Law-of-Total-Probability, Independence, and the downstream users Naive-Bayes-Classifier, Chain-Rule-of-Probability, Mutual-Information.

The one picture behind every trap

Before the questions, fix the mental image. Draw the whole set of possible outcomes as a rectangle of area $1$ . Event $A$ is a blob inside it; event $B$ is another blob; where they overlap is $A \cap B$ .

Figure s01 — the area picture of conditioning. A white rectangle is the whole sample space (total area $1$ ). The lavender circle is event $A$ , the coral circle is event $B$ , and their lens-shaped overlap is $A \cap B$ with area $P (A \cap B)$ . Conditioning on $B$ means erasing everything outside the coral circle and re-asking what fraction of that circle is also lavender — i.e. dividing the overlap area by the coral circle's area $P (B)$ . The caption line at the bottom states $P (A ∣ B) = P (A \cap B) / P (B)$ .

Every question below is really a question about which blob you renormalise by. Keep the rectangle in mind.

True or false — justify

True or false:

P (A ∣ B)

is always less than or equal to

P (A)

because conditioning "shrinks the universe".

False. Shrinking the universe can raise the probability — if

B

overlaps heavily with

A

, learning

B

makes

A

more likely (e.g. "free" raises

P (spam)

from 60% to 92%).

True or false:

P (A ∣ B) + P (A ∣ \neg B) = 1

False. What sums to

1

P (A ∣ B) + P (\neg A ∣ B)

, because both live in the same conditioning world

B

A ∣ B

and

A ∣ \neg B

live in two different universes and need not add to anything.

True or false: if

A

and

B

are independent, then

P (A ∣ B) = P (B ∣ A)

False. Independence gives

P (A ∣ B) = P (A)

and

P (B ∣ A) = P (B)

, and

P (A)

need not equal

P (B)

. Independence means "no influence", not "symmetric numbers".

True or false:

P (A ∣ B) = P (B ∣ A)

can only happen by coincidence.

False. Bayes shows they are equal exactly when the priors match:

P (A ∣ B) = P (B ∣ A)

iff

P (A) = P (B)

(given both are non-zero). Equal priors is a real, common condition, not luck.

True or false: if

P (A ∣ B) = P (A)

then also

P (B ∣ A) = P (B)

True. Independence is symmetric:

P (A ∣ B) = P (A)

rearranges to

P (A \cap B) = P (A) P (B)

, which is symmetric in

A

and

B

, so it flips back to

P (B ∣ A) = P (B)

. See Independence.

True or false:

P (A \cap B) = P (A ∣ B) P (B)

holds for any events with

P (B) > 0

, independent or not.

True. This is the multiplication rule, just the definition rearranged. Independence is a special case where

P (A ∣ B)

simplifies to

P (A)

; the rule itself never needs independence. See Chain-Rule-of-Probability.

True or false: mutually exclusive events

A, B

(with positive probability) are also independent.

False, and they're the opposite. If

A, B

are mutually exclusive then

B

happening tells you

A

definitely did not — that is maximal dependence, since

P (A ∣ B) = 0 \neq = P (A)

True or false: the Law of Total Probability needs its partition pieces (a set of mutually exclusive, exhaustive events

B_{1}, \dots, B_{n}

that together cover the whole sample space) to be equally likely.

False. It needs them only to be such a partition — mutually exclusive and exhaustive. Each pathway is weighted by its own

P (B_{i})

, so unequal weights are the normal case. See Law-of-Total-Probability.

True or false: a positive test on a 95%-accurate test means 95% chance of disease.

False — this is base-rate neglect. The posterior also depends on the disease's rarity; a rare disease drags the true positive probability far below the test's accuracy.

Spot the error

"A friend claims:

P (A ∣ B) = \frac{P ( A \cap B )}{P ( A )}

." — what's wrong?

The denominator must be

P (B)

, the conditioning event, not

P (A)

. You renormalize by the universe you were told you're in, and that universe is

B

"Because 80% of spam contains 'free', an email with 'free' is 80% likely to be spam." — spot the error.

That is

P (F ∣ S) = 0.8

, but the question asks

P (S ∣ F)

. Flipping the conditioning needs Bayes and the priors of spam vs ham; the two numbers are different quantities.

P (A \cap B \cap C) = P (A) P (B) P (C)

, so the three events are independent." — what's missing?

The single joint factorisation of the triple product is not enough on its own. Full independence also requires every pair to factor separately —

P (A \cap B) = P (A) P (B)

, and likewise for

A, C

and

B, C

. It is possible for the triple to factor while a pair does not, so you must check the pairwise conditions too.

"Since

B_{1}, B_{2}

don't overlap,

P (A) = P (A ∣ B_{1}) + P (A ∣ B_{2})

." — spot the error.

You forgot the weights. Here

B_{1}, B_{2}

are the two exclusive, exhaustive partition pieces; the Law of Total Probability is

P (A) = P (A ∣ B_{1}) P (B_{1}) + P (A ∣ B_{2}) P (B_{2})

. Dropping the

P (B_{i})

factors mixes fractions-of-different-universes with no common scale.

"We compared two hypotheses: both give

P (data ∣ H_{i}) = 0.3

, so each hypothesis is 30% likely." — what's wrong?

0.3

is a likelihood

P (data ∣ H_{i})

, not a posterior

P (H_{i} ∣ data)

. To get posteriors you multiply by priors and normalize by the total; unnormalized likelihoods only rank, they aren't probabilities.

"In the chain rule I wrote

P (A_{1} \cap A_{2} \cap A_{3}) = P (A_{1}) P (A_{2} ∣ A_{1}) P (A_{3} ∣ A_{2})

." — spot the error.

The last factor must condition on everything before it:

P (A_{3} ∣ A_{1} \cap A_{2})

, not just

A_{2}

. Only under a Markov assumption may you drop earlier terms — that's exactly the shortcut Hidden-Markov-Models make on purpose. See Chain-Rule-of-Probability.

"The test's false-positive rate is 10%, so

P (healthy ∣ positive) = 0.10

." — what's wrong?

10%

P (positive ∣ healthy)

, a property of the test on healthy people.

P (healthy ∣ positive)

is the reversed conditioning and, for a rare disease, is actually large.

Why questions

Why do we divide by

P (B)

instead of just using the overlap

P (A \cap B)

Because once you know

B

happened,

B

is the new "whole" and must weigh

1

. Dividing rescales the overlap so probabilities inside the shrunken universe sum to

1

again.

Why does Bayes' theorem "flip" the conditioning, and why is that useful?

We often measure

P (evidence ∣ cause)

(a test's behaviour) but want

P (cause ∣ evidence)

(a diagnosis). Bayes converts the direction we can measure into the direction we need. See Bayes-Theorem.

Why must

P (B) > 0

appear as a condition in the definition of

P (A ∣ B)

You'd be dividing by zero — and conceptually, "given an impossible event" has no meaning because there are no

B

-outcomes to renormalize over.

Why does a very rare disease make a positive test so unconvincing?

The healthy majority is enormous, so even a small false-positive rate produces many false positives in raw count, swamping the few true positives from the tiny sick group.

Why is independence a modelling assumption in Naive Bayes rather than a fact?

Words in real emails are correlated, so features aren't truly independent; Naive Bayes pretends they are to make the product of conditionals tractable, trading exactness for speed. See Naive-Bayes-Classifier.

Why does mutual information measure "how much conditioning changes things"?

It is zero exactly when

P (A ∣ B) = P (A)

for all outcomes — i.e. under independence — and grows as knowing one variable shifts the probabilities of the other. See Mutual-Information.

Why can you factor

P (A_{1} \cap \dots \cap A_{n})

in any order you like?

Because the joint probability is a single fixed number; the chain rule just decomposes it, and every ordering of the conditionals recomposes the same value. Convenience (which conditionals you know) decides the order.

Edge cases

P (B) = 1

(a certain event), what is

P (A ∣ B)

It equals

P (A)

. Conditioning on something that always happens tells you nothing new, so the universe doesn't shrink.

A \subseteq B

(A implies B), what is

P (A ∣ B)

and

P (B ∣ A)

P (B ∣ A) = 1

(if

A

happens,

B

must too), while

P (A ∣ B) = P (A) / P (B) \leq 1

— the two directions are wildly asymmetric here, a clean illustration of why

P (A ∣ B) \neq = P (B ∣ A)

A

and

B

are the same event, what is

P (A ∣ B)

It is

1

: given

A

happened, the probability

A

happened is certain. The overlap

P (A \cap A) = P (A)

divided by

P (A)

gives one.

What happens to

P (D ∣ T^{+})

in the medical example as prevalence

P (D) \to 0

The posterior

\to 0

as well: with essentially no sick people, virtually every positive is a false positive, so a positive test carries almost no information.

What happens to

P (D ∣ T^{+})

as the test becomes perfect (false-positive rate

\to 0

The posterior

\to 1

: if only truly sick people can test positive, a positive result is conclusive regardless of how rare the disease is.

Can

P (A ∣ B)

be defined when

A

is impossible, i.e.

P (A) = 0

Yes — it's simply

0

, since

P (A \cap B) \leq P (A) = 0

. Only the conditioning event

B

must have positive probability; the event

A

can be anything.

Recall One-line summary of the traps

Conditional-probability mistakes almost always reduce to three moves: (1) flipping $P (A ∣ B)$ and $P (B ∣ A)$ , (2) forgetting the prior $P (B_{i})$ weights, and (3) dividing by the wrong universe. Catch those three and most traps disappear.