Independence and mutual exclusivity: Exercises

This page is a graded ladder. Each rung asks more of you than the last. Do the problem first (pen and paper), then open the collapsible solution. Every symbol used here was built in the parent note Independence and mutual exclusivity — if a piece feels shaky, revisit 1.3.01-Basic-probability-concepts, 1.3.02-Conditional-probability, or 1.3.03-Bayes-theorem before continuing.

Figure — Independence and mutual exclusivity

The picture above is your compass for the whole page: two circles inside the box of all outcomes (the sample space). Look at the left panel: the magenta $A$ circle and violet $B$ circle touch nowhere — that empty gap between them is what $P (A \cap B) = 0$ looks like, so these events are mutually exclusive. Now the right panel: the circles do overlap, and when that shared lens area equals the product $P (A) P (B)$ , the events are independent. Keep flicking your eye between the two panels — they are different pictures, and for non-trivial events they are never the same one. Every exercise below is really just asking "which panel am I in?".

Level 1 — Recognition

Goal: read a scenario and name which property (if any) holds, straight from the definitions.

Exercise 1.1

A fair six-sided die is rolled once. Let $A =$ "roll an even number" and $B =$ "roll a 3". Are $A$ and $B$ mutually exclusive? Are they independent?

Recall Solution 1.1

List the outcomes. The sample space is ${1, 2, 3, 4, 5, 6}$ , each with probability $\frac{1}{6}$ .

$A = {2, 4, 6}$ , so $P (A) = \frac{3}{6} = \frac{1}{2}$ .
$B = {3}$ , so $P (B) = \frac{1}{6}$ .

Mutually exclusive? (WHAT: find $A \cap B$ .) The overlap $A \cap B$ is outcomes that are both even and equal to 3. There are none — 3 is odd. So $P (A \cap B) = 0$ . This is the left panel of the figure — no overlap. Yes, mutually exclusive.

Independent? (WHAT: compare $P (A \cap B)$ to $P (A) P (B)$ .) $P (A) \cdot P (B) = \frac{1}{2} \cdot \frac{1}{6} = \frac{1}{12} \neq = 0 = P (A \cap B) .$ Not independent. (As the parent note proved: non-trivial mutually exclusive events can never be independent.)

Exercise 1.2

Two fair coins are flipped. $A =$ "first coin is heads", $B =$ "second coin is heads". Mutually exclusive? Independent?

Recall Solution 1.2

Sample space: ${H H, H T, T H, T T}$ , each probability $\frac{1}{4}$ .

$A = {H H, H T}$ , $P (A) = \frac{1}{2}$ .
$B = {H H, T H}$ , $P (B) = \frac{1}{2}$ .
$A \cap B = {H H}$ , $P (A \cap B) = \frac{1}{4}$ .

Mutually exclusive? $P (A \cap B) = \frac{1}{4} \neq = 0$ . No — both coins can be heads. This is the right panel: the circles overlap. Independent? $P (A) P (B) = \frac{1}{2} \cdot \frac{1}{2} = \frac{1}{4} = P (A \cap B)$ . Yes. The flips are separate physical processes; one tells you nothing about the other.

Level 2 — Application

Goal: use the multiplication and union rules to get numbers.

Exercise 2.1

A card is drawn from a standard 52-card deck, replaced, the deck reshuffled, and a second card drawn. Let $A =$ "first card is an ace", $B =$ "second card is a King". Find $P (A \cap B)$ .

Recall Solution 2.1

WHY independence applies: replacement puts the deck back to 52 cards, so the second draw's probabilities are unchanged by the first. The two draws are independent.

$P (A) = \frac{4}{52} = \frac{1}{13}$ (4 aces).
$P (B) = \frac{4}{52} = \frac{1}{13}$ (4 Kings).

By the independence multiplication rule: $P (A \cap B) = P (A) \cdot P (B) = \frac{1}{13} \cdot \frac{1}{13} = \frac{1}{169} \approx 0.00592.$

Exercise 2.2

On a single die roll, $A =$ "roll a 1", $B =$ "roll a 2", $C =$ "roll a 3". All are pairwise mutually exclusive. Find $P (A \cup B \cup C)$ .

Recall Solution 2.2

WHY the simple sum works: for mutually exclusive events, every pairwise overlap is $0$ (the left panel picture), so the union formula loses all subtraction terms and collapses to plain addition. $P (A \cup B \cup C) = P (A) + P (B) + P (C) = \frac{1}{6} + \frac{1}{6} + \frac{1}{6} = \frac{3}{6} = \frac{1}{2} .$

Exercise 2.3

Two fair dice are rolled. $A =$ "first die shows 4", $B =$ "second die shows 4". Find $P (A \cup B)$ ("at least one die shows a 4").

Recall Solution 2.3

These are independent (separate dice), not mutually exclusive — outcome $(4, 4)$ is in both. This is the right panel: the circles overlap.

$P (A) = \frac{6}{36} = \frac{1}{6}$ , $P (B) = \frac{1}{6}$ .
$P (A \cap B) = P (A) P (B) = \frac{1}{6} \cdot \frac{1}{6} = \frac{1}{36}$ .

Use the full union formula (WHY: overlap is non-zero, so we must subtract it once): $P (A \cup B) = P (A) + P (B) - P (A \cap B) = \frac{6}{36} + \frac{6}{36} - \frac{1}{36} = \frac{11}{36} \approx 0.3056.$

Level 3 — Analysis

Goal: probe why a relationship holds by testing, not by intuition.

Exercise 3.1

Two fair dice are rolled. $A =$ "first die shows 4", $C =$ "the sum is 8". Determine whether $A$ and $C$ are independent, and quantify how much knowing $A$ changes the belief in $C$ .

Recall Solution 3.1

Enumerate.

$P (A) = \frac{6}{36} = \frac{1}{6}$ .
Sums to 8: $(2, 6), (3, 5), (4, 4), (5, 3), (6, 2)$ → $P (C) = \frac{5}{36}$ .
$A \cap C$ : first die is 4 and sum is 8 → only $(4, 4)$ → $P (A \cap C) = \frac{1}{36}$ .

Independence test: $P (A) \cdot P (C) = \frac{1}{6} \cdot \frac{5}{36} = \frac{5}{216} \approx 0.0231,$ while $P (A \cap C) = \frac{1}{36} \approx 0.0278$ . Since $\frac{5}{216} \neq = \frac{1}{36}$ , not independent.

Quantify the information gain: $P (C ∣ A) = \frac{P ( A \cap C )}{P ( A )} = \frac{1/36}{1/6} = \frac{1}{6} \approx 0.1667,$ versus the prior $P (C) = \frac{5}{36} \approx 0.1389$ . Knowing the first die is 4 raises the sum-is-8 belief from $0.139$ to $0.167$ . Non-zero change ⇒ dependence confirmed.

Exercise 3.2

Same two dice. $A =$ "first die shows 4", $D =$ "the sum is 7". Are $A$ and $D$ independent?

Recall Solution 3.2

$P (A) = \frac{1}{6}$ .
Sums to 7: $(1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1)$ → $P (D) = \frac{6}{36} = \frac{1}{6}$ .
$A \cap D$ : first die 4 and sum 7 → $(4, 3)$ only → $P (A \cap D) = \frac{1}{36}$ .

Test: $P (A) P (D) = \frac{1}{6} \cdot \frac{1}{6} = \frac{1}{36} = P (A \cap D)$ . Independent!

WHY this differs from 3.1: a sum of 7 is reachable from every first-die value (there is always exactly one matching second die). So learning the first die never tips the odds of "sum = 7". A sum of 8 has no partner for a first die of 1, so it is not reachable uniformly — hence dependence. Same dice, opposite verdicts, decided purely by the arithmetic.

Level 4 — Synthesis

Goal: combine conditional probability, Bayes, and independence into one chain.

Exercise 4.1 (Conditional independence)

A patient either has the flu ( $C$ , with $P (C) = 0.10$ ) or not. Given flu, the chance of coughing is $P (cough ∣ C) = 0.8$ and of fever is $P (fever ∣ C) = 0.6$ ; given no flu, $P (cough ∣ \neg C) = 0.1$ and $P (fever ∣ \neg C) = 0.05$ . Assume cough and fever are conditionally independent given flu status. A patient has both cough and fever. Find $P (C ∣ cough \cap fever)$ .

Recall Solution 4.1

Step 1 — WHY conditional independence helps. It lets us split a joint conditional into a product: $P (cough \cap fever ∣ C) = P (cough ∣ C) \cdot P (fever ∣ C) .$ (This is the Naive-Bayes trick from 3.1.01-Naive-Bayes-classifier — see also 5.2.03-Probabilistic-graphical-models.)

Step 2 — likelihoods.

With flu: $0.8 \times 0.6 = 0.48$ .
Without flu: $0.1 \times 0.05 = 0.005$ .

Step 3 — Bayes' theorem (from 1.3.03-Bayes-theorem). Let $E = cough \cap fever$ . $P (C ∣ E) = \frac{P ( E ∣ C ) P ( C )}{P ( E ∣ C ) P ( C ) + P ( E ∣ \neg C ) P ( \neg C )} .$ WHY this denominator? The patient is definitely in exactly one of two worlds — flu or no-flu — and these two cases are mutually exclusive and exhaustive. So the total probability of seeing the evidence $E$ is the evidence-weight from the flu world plus the evidence-weight from the no-flu world. That sum is precisely the denominator: it re-normalizes the numerator into a genuine probability by dividing by "all the ways $E$ could have happened". Numerator: $0.48 \times 0.10 = 0.048$ . Denominator: $0.048 + 0.005 \times 0.90 = 0.048 + 0.0045 = 0.0525$ . $P (C ∣ E) = \frac{0.048}{0.0525} \approx 0.9143.$ So seeing both symptoms lifts flu belief from a $10%$ prior to about $91.4%$ .

Exercise 4.2

Show that if $A$ and $B$ are independent, then $A$ and $B^{c}$ (the complement of $B$ , i.e. " $B$ does not happen") are also independent.

Recall Solution 4.2

WHAT we want: $P (A \cap B^{c}) = P (A) \cdot P (B^{c})$ . Step 1 — split $A$ . Every outcome in $A$ either lands in $B$ or in $B^{c}$ , and these are exclusive, so $P (A) = P (A \cap B) + P (A \cap B^{c}) .$ Step 2 — solve for the piece we want. $P (A \cap B^{c}) = P (A) - P (A \cap B) = P (A) - P (A) P (B) (used independence) .$ Step 3 — factor. $P (A \cap B^{c}) = P (A) (1 - P (B)) = P (A) \cdot P (B^{c}),$ since $P (B^{c}) = 1 - P (B)$ . This is exactly the independence definition for $A$ and $B^{c}$ . $■$ Takeaway: " $A$ ignores $B$ " automatically means " $A$ ignores not- $B$ ".

Level 5 — Mastery

Goal: build a counterexample and reason about the boundary between the two concepts.

Exercise 5.1 (Pairwise vs. mutual independence)

A fair coin is flipped twice. Define:

$X =$ "first flip is heads",
$Y =$ "second flip is heads",
$Z =$ "the two flips match (both H or both T)".

Show that $X, Y, Z$ are pairwise independent but not mutually independent.

Recall Solution 5.1

Sample space ${H H, H T, T H, T T}$ , each $\frac{1}{4}$ .

$X = {H H, H T}$ , $Y = {H H, T H}$ , $Z = {H H, T T}$ . Each has probability $\frac{1}{2}$ .

Pairwise checks:

$X \cap Y = {H H}$ : $P = \frac{1}{4} = \frac{1}{2} \cdot \frac{1}{2}$ . ✓
$X \cap Z = {H H}$ : $P = \frac{1}{4} = \frac{1}{2} \cdot \frac{1}{2}$ . ✓
$Y \cap Z = {H H}$ : $P = \frac{1}{4} = \frac{1}{2} \cdot \frac{1}{2}$ . ✓ All three pairs are independent.

Mutual (triple) check: $P (X \cap Y \cap Z) = P ({H H}) = \frac{1}{4},$ but $P (X) P (Y) P (Z) = \frac{1}{2} \cdot \frac{1}{2} \cdot \frac{1}{2} = \frac{1}{8}$ . Since $\frac{1}{4} \neq = \frac{1}{8}$ , not mutually independent.

WHY: once you know $X$ and $Y$ , you already know $Z$ (if both heads, they match). So $Z$ carries no fresh randomness given the other two — a hidden dependence invisible to any single pair.

Exercise 5.2 (Boundary reasoning)

Suppose $A$ and $B$ satisfy both $P (A \cap B) = 0$ (mutually exclusive) and $P (A \cap B) = P (A) P (B)$ (independent). What must be true of $A$ and $B$ ? Interpret.

Recall Solution 5.2

Setting the two conditions equal: $P (A) P (B) = 0$ . A product is zero only if a factor is zero, so $P (A) = 0 or P (B) = 0.$ Interpretation: the only way an event pair can be simultaneously mutually exclusive and independent is if at least one event is effectively impossible (probability $0$ ). For any two "real" events with positive probability, the two properties are mutually incompatible — precisely the "opposites" claim from the parent note, now proved as a hard boundary.

Recall Self-test recap (one line each)

Non-trivial mutually exclusive events can be independent? ::: No — for positive-probability events the two are incompatible (Ex 5.2). $P (A \cup B)$ when $A, B$ overlap? ::: $P (A) + P (B) - P (A \cap B)$ ; subtract the overlap once (Ex 2.3). What unlocks the product $\prod_{i} P (x_{i} ∣ y)$ in Naive Bayes? ::: Conditional independence given the class $y$ (Ex 4.1). Does pairwise independence imply mutual independence? ::: No — see the coin-match counterexample (Ex 5.1).