Conditional probability: Exercises — AI-ML deep dive

This is your practice ladder for conditional probability. Every rung is harder than the last:

L1 Recognition — can you spot which formula applies?
L2 Application — plug into one formula cleanly.
L3 Analysis — combine several ideas, watch the base rate.
L4 Synthesis — chain rules, multi-step reasoning, Bayes across many hypotheses.
L5 Mastery — build something new (a mini classifier, a limiting case).

Every solution is hidden inside a collapsible callout so you can test yourself first. Try the problem, then reveal.

Before we start, one picture to keep in your head the whole way down: conditioning means shrinking the universe to a smaller box and re-measuring inside it.

L1 — Recognition

Exercise 1.1 (L1 Recognition)

A jar has 10 tickets numbered 1–10. You draw one. Let $A = {$ number is even $}$ and $B = {$ number $> 5}$ . Someone tells you the number is greater than 5. What is $P (A ∣ B)$ ?

Recall Solution

What we know: $B = {6, 7, 8, 9, 10}$ , so the shrunken universe has $5$ outcomes. Inside it, the even ones are ${6, 8, 10}$ , so $3$ favourable. Why this is just counting: all tickets are equally likely, so probability = (favourable) / (size of universe). Conditioning on $B$ replaces the size-10 universe with a size-5 universe. $P (A ∣ B) = \frac{∣ A \cap B ∣}{∣ B ∣} = \frac{3}{5} = 0.6$ Answer: $== 0.6 ==$ .

Exercise 1.2 (L1 Recognition)

You are given $P (A \cap B) = 0.2$ and $P (B) = 0.5$ . Which single formula gives $P (A ∣ B)$ , and what is its value?

Recall Solution

The definition of conditional probability applies directly — no Bayes, no total probability needed. $P (A ∣ B) = \frac{P ( A \cap B )}{P ( B )} = \frac{0.2}{0.5} = 0.4$ Answer: $== 0.4 ==$ .

Exercise 1.3 (L1 Recognition)

True or false: "If $P (A ∣ B) = P (A)$ , then $A$ and $B$ are independent." Justify in one line.

Recall Solution

True. $P (A ∣ B) = P (A)$ means learning $B$ did not move the probability of $A$ — that is exactly the definition of independence. Equivalently $P (A \cap B) = P (A) P (B)$ .

L2 — Application

Exercise 2.1 (L2 Application)

A factory has two machines. Machine X makes $70%$ of parts, Machine Y makes $30%$ . X's defect rate is $2%$ ; Y's is $5%$ . Pick a random part — what is $P (defective)$ ?

Recall Solution

This is the law of total probability: a part is defective through one of two disjoint "pathways" (came from X, or came from Y). $P (def) = P (def ∣ X) P (X) + P (def ∣ Y) P (Y)$ $= 0.02 \times 0.70 + 0.05 \times 0.30 = 0.014 + 0.015 = 0.029$ Answer: $== 0.029 ==$ (i.e. $2.9%$ ).

Exercise 2.2 (L2 Application)

Same factory. A part is found defective. What is the probability it came from Machine Y?

Recall Solution

Now we flip the conditioning with Bayes' theorem. We already computed the denominator $P (def) = 0.029$ in 2.1. $P (Y ∣ def) = \frac{P ( def ∣ Y ) P ( Y )}{P ( def )} = \frac{0.05 \times 0.30}{0.029} = \frac{0.015}{0.029} \approx 0.5172$ Answer: $\approx== 0.517 ==$ (about $51.7%$ ). Even though Y makes only $30%$ of parts, it produces more than half the defectives because its defect rate is higher.

Exercise 2.3 (L2 Application)

Two cards are drawn without replacement from a standard 52-card deck. What is $P (both aces)$ ?

Recall Solution

Use the multiplication (chain) rule because the draws are sequential and the second depends on the first. $P (A_{1} \cap A_{2}) = P (A_{1}) P (A_{2} ∣ A_{1}) = \frac{4}{52} \times \frac{3}{51} = \frac{12}{2652} = \frac{1}{221}$ Answer: $\frac{1}{221} \approx== 0.004525 ==$ .

L3 — Analysis

Exercise 3.1 (L3 Analysis — base rate)

A rare disease affects $1$ in $1000$ people. A test has sensitivity $99%$ and specificity $98%$ . You test positive. What is the probability you actually have the disease?

Recall Solution

Events: $D$ (disease), $P (D) = 0.001$ , $P (\neg D) = 0.999$ . $P (T^{+} ∣ D) = 0.99$ . Specificity $0.98 \Rightarrow$ false-positive rate $P (T^{+} ∣ \neg D) = 0.02$ . Step 1 — total probability of a positive test: $P (T^{+}) = 0.99 \times 0.001 + 0.02 \times 0.999 = 0.00099 + 0.01998 = 0.02097$ Step 2 — Bayes: $P (D ∣ T^{+}) = \frac{0.99 \times 0.001}{0.02097} = \frac{0.00099}{0.02097} \approx 0.0472$ Answer: $\approx== 0.0472 ==$ (about $4.7%$ ). The disease is so rare that most positives come from the huge healthy majority.

Exercise 3.2 (L3 Analysis)

In the setup of 3.1, how many people out of $100, 000$ tested would you expect to be true positives vs false positives? Confirm the posterior from these counts.

Recall Solution

Out of $100, 000$ : diseased $= 0.001 \times 100000 = 100$ ; healthy $= 99, 900$ .

True positives: $100 \times 0.99 = 99$ .
False positives: $99, 900 \times 0.02 = 1998$ .
Total positives: $99 + 1998 = 2097$ . $P (D ∣ T^{+}) = \frac{99}{2097} \approx 0.0472$ Matches 3.1. Why counting works: frequencies are the probabilities scaled by $N$ ; the $N$ cancels in the ratio.

Exercise 3.3 (L3 Analysis)

Show that if $A$ and $B$ are independent, then $A$ and $\neg B$ (not- $B$ ) are also independent.

Recall Solution

Independence gives $P (A \cap B) = P (A) P (B)$ . Split $A$ over $B$ and $\neg B$ : $P (A) = P (A \cap B) + P (A \cap \neg B) \Rightarrow P (A \cap \neg B) = P (A) - P (A) P (B) = P (A) (1 - P (B)) = P (A) P (\neg B) .$ That last equality is exactly the independence condition for $A$ and $\neg B$ . $■$

L4 — Synthesis

Exercise 4.1 (L4 Synthesis — chain rule)

An urn has $3$ red and $2$ green balls. Draw $3$ without replacement. What is $P (Red, Green, Red)$ in that order?

Recall Solution

Apply the chain rule step by step, updating the urn each time. $P (R_{1}) = \frac{3}{5}, P (G_{2} ∣ R_{1}) = \frac{2}{4}, P (R_{3} ∣ R_{1} G_{2}) = \frac{2}{3}$ $P (R GR) = \frac{3}{5} \cdot \frac{2}{4} \cdot \frac{2}{3} = \frac{12}{60} = \frac{1}{5} = 0.2$ Answer: $== 0.2 ==$ .

Exercise 4.2 (L4 Synthesis — three hypotheses)

Three biased coins: A (heads $0.9$ ), B (heads $0.5$ ), C (heads $0.1$ ). You pick one at random (equal chance) and flip it — it lands heads. What is the probability it was coin A?

Recall Solution

Priors $P (A) = P (B) = P (C) = \frac{1}{3}$ . Total probability of heads: $P (H) = \frac{1}{3} (0.9) + \frac{1}{3} (0.5) + \frac{1}{3} (0.1) = \frac{1}{3} (1.5) = 0.5$ Bayes for coin A: $P (A ∣ H) = \frac{P ( H ∣ A ) P ( A )}{P ( H )} = \frac{0.9 \times \frac{1}{3}}{0.5} = \frac{0.3}{0.5} = 0.6$ Answer: $== 0.6 ==$ .

Exercise 4.3 (L4 Synthesis — sequential Bayes)

Same three coins. You flip the chosen coin twice and get heads then heads. Now what is $P (A ∣ H H)$ ? (Assume flips of a fixed coin are independent.)

Recall Solution

Likelihood of $H H$ for each coin is (heads prob) $^{2}$ : A $= 0.81$ , B $= 0.25$ , C $= 0.01$ . $P (H H) = \frac{1}{3} (0.81 + 0.25 + 0.01) = \frac{1}{3} (1.07) = 0.356 \overline{6}$ $P (A ∣ H H) = \frac{0.81 \times \frac{1}{3}}{0.356 6} = \frac{0.27}{0.356 6} \approx 0.7570$ Answer: $\approx== 0.757 ==$ . Two heads is stronger evidence for the heads-heavy coin than one — the posterior climbed from $0.6$ to $\approx 0.757$ . This is Bayesian updating in action.

L5 — Mastery

Exercise 5.1 (L5 Mastery — mini Naive Bayes)

A spam filter treats words as conditionally independent given the class (the naive Bayes assumption). Data: $P (S) = 0.6$ , $P (H) = 0.4$ . For spam: $P ("free" ∣ S) = 0.8$ , $P ("win" ∣ S) = 0.5$ . For ham: $P ("free" ∣ H) = 0.1$ , $P ("win" ∣ H) = 0.2$ . An email contains both "free" and "win". Compute $P (S ∣ free, win)$ .

Recall Solution

Naive assumption: $P (free, win ∣ S) = P (free ∣ S) P (win ∣ S)$ (words independent given class). Numerator (spam pathway): $0.8 \times 0.5 \times 0.6 = 0.24$ . Ham pathway: $0.1 \times 0.2 \times 0.4 = 0.008$ . Evidence: $P (free, win) = 0.24 + 0.008 = 0.248$ . $P (S ∣ free, win) = \frac{0.24}{0.248} \approx 0.9677$ Answer: $\approx== 0.968 ==$ (about $96.8%$ ). Two spam-indicative words push the posterior even higher than either alone.

Exercise 5.2 (L5 Mastery — degenerate / limiting case)

In Exercise 5.1, suppose the word "win" never appears in ham, i.e. $P ("win" ∣ H) = 0$ . Recompute $P (S ∣ free, win)$ and explain why this is dangerous in a real classifier.

Recall Solution

Ham pathway becomes $0.1 \times 0 \times 0.4 = 0$ . So evidence $= 0.24 + 0 = 0.24$ and $P (S ∣ free, win) = \frac{0.24}{0.24} = 1.0$ The model is now 100% certain it's spam — a single zero probability vetoes the entire ham hypothesis, no matter what other evidence says. In practice one unseen word could wrongly zero out a class. Fix in real systems: Laplace smoothing adds a small count so no likelihood is exactly $0$ . Certainty should never come from a single missing observation.

Exercise 5.3 (L5 Mastery — build the boundary)

Using 5.1's spam model but ignoring "win", find the prior $P (S) = p$ at which seeing only "free" makes spam and ham equally likely, i.e. $P (S ∣ free) = 0.5$ .

Recall Solution

Equal posteriors $\Leftrightarrow$ equal weighted likelihoods: $P (free ∣ S) p = P (free ∣ H) (1 - p)$ $0.8 p = 0.1 (1 - p) \Rightarrow 0.8 p = 0.1 - 0.1 p \Rightarrow 0.9 p = 0.1 \Rightarrow p = \frac{1}{9} \approx 0.1111$ Answer: $p = \frac{1}{9} \approx== 0.1111 ==$ . Below this prior, "free" alone is not enough to call an email spam — the decision boundary depends on the base rate, exactly the L3 lesson resurfacing.

Recall One-line self-check

Every problem here reduced to one of three moves — which three? ::: (1) definition $P (A \cap B) / P (B)$ , (2) law of total probability to build the denominator, (3) Bayes to flip the conditioning.

Where this leads

These same moves power Naive Bayes, Hidden Markov Models, Bayesian inference, and show up in mutual information. Master the ladder and every one of those becomes "just conditioning, repeated".