Exercises — Bayes' theorem — derivation and applications

The sentence fixes the actual weather first ("when it actually rains") and then reports how the app behaves. So the given (the condition, the thing after the bar) is $R$. That makes it $P(F\mid R)=\mathrm{0.90.}$ This is a likelihood — probability of the evidence (forecast) given the cause (rain). The quantity a user actually wants, "it forecast rain, so how likely is real rain?", is the reversed one $P(R\mid F)$, and Bayes is what connects them.

• (a) is the prior $P(D)=0.01$ — belief before any test.
• (b) is the likelihood $P(+\mid D)=0.99$ — how the test behaves given you are sick (sensitivity is a "true-positive rate").
• (c) is the posterior $P(D\mid +)$ — the updated belief after seeing the positive. This is the unknown Bayes computes.

Yes: $0.7+0.3=1$. A partition means the causes are mutually exclusive (a marble is in exactly one bag) and exhaustive (there are no other bags). Every marble is accounted for exactly once, so the priors must sum to $1$. If they did not, the denominator in Bayes could never normalise the posteriors to $1$.

Priors: $P(A)=0.8,P(B)=0.2$. Likelihoods: $P(\text{pass}\mid A)=0.95,P(\text{pass}\mid B)=0.60$.

Evidence (Law of Total Probability — a pass can come from A or B): $P(\text{pass})=(0.95)(0.8)+(0.60)(0.2)=0.76+0.12=\mathrm{0.88.}$ Bayes: $P(A\mid \text{pass})=\frac{0.76}{0.88}=\frac{19}{22}\approx \mathrm{0.8636.}$ The posterior ($0.864$) barely rises above the prior ($0.8$) — passing is only mild evidence of being Type-A because Type-A items pass almost as easily as Type-B.

Fail likelihoods are the complements: $P(\text{fail}\mid A)=1-0.95=0.05$, $P(\text{fail}\mid B)=1-0.60=0.40$.

Evidence: $P(\text{fail})=(0.05)(0.8)+(0.40)(0.2)=0.04+0.08=\mathrm{0.12.}$ (Check: $0.88+0.12=1$, so passes and fails partition all items.)

Bayes: $P(B\mid \text{fail})=\frac{0.08}{0.12}=\frac{2}{3}\approx \mathrm{0.6667.}$ A failure is strong evidence of Type-B: even though only $20\%$ of items are Type-B, they fail eight times as often, so they dominate the failures.

$P(+)=(0.99)(0.01)+(0.01)(0.99)=0.0099+0.0099=\mathrm{0.0198.}$ $P(D\mid +)=\frac{0.0099}{0.0198}=\mathrm{0.5.}$ Cutting the false-positive rate from $5\%$ to $1\%$ lifts the posterior from $16.7\%$ (parent note) to exactly $50\%$. Specificity, not just sensitivity, is what tames false positives on a rare disease.

Joints $P(D\mid \cdot )P(\cdot )$: $X:(0.01)(0.50)=0.005,Y:(0.02)(0.30)=0.006,Z:(0.04)(0.20)=\mathrm{0.008.}$ Evidence: $P(D)=0.005+0.006+0.008=\mathrm{0.019.}$ Posteriors: $P(X\mid D)=\frac{0.005}{0.019}\approx 0.2632,P(Y\mid D)=\frac{0.006}{0.019}\approx 0.3158,P(Z\mid D)=\frac{0.008}{0.019}\approx \mathrm{0.4211.}$ They sum to $1$ (good). Despite making the fewest chips, Z is the most likely source of a defect because its defect rate is four times X's. See the figure: the prior bars (widths) shrink toward the posterior once we weight by defect rate.

Reasoning first: halving Z's defect rate shrinks Z's joint $P(D\mid Z)P(Z)$ from $0.008$ to $0.004$, while X and Y are unchanged. A smaller numerator with a smaller total pulls Z's share down.

Verify: new joints $X=0.005,Y=0.006,Z=0.004$; evidence $=0.015$. $P(Z\mid D)=\frac{0.004}{0.015}\approx 0.2667,$ down from $0.4211$. Y now becomes the leader ($0.006/0.015=0.4$). The most likely cause is not fixed — it tracks the product of prior and likelihood.

Priors: $P(S)=0.4,P(H)=0.6$. Likelihoods: $P(\text{free}\mid S)=0.6,P(\text{free}\mid H)=0.1$. $P(\text{free})=(0.6)(0.4)+(0.1)(0.6)=0.24+0.06=\mathrm{0.30.}$ $P(S\mid \text{free})=\frac{0.24}{0.30}=\mathrm{0.8.}$ The ham fraction is the complement, $P(H\mid \text{free})=0.06/0.30=0.2$. This "one word, one update" step is exactly the atom of a Naive Bayes Classifier, which multiplies many such word-updates together.

Conditional independence means, given a fixed true state, the two results multiply: $P(++\mid D)=0.99^2=0.9801,P(++\mid \bar{D})=0.05^2=\mathrm{0.0025.}$ Evidence: $P(++)=(0.9801)(0.01)+(0.0025)(0.99)=0.009801+0.002475=\mathrm{0.012276.}$ Bayes: $P(D\mid ++)=\frac{0.009801}{0.012276}\approx \mathrm{0.7984.}$ One positive gave only $16.7\%$; a second independent positive rockets it to $\approx 79.8\%$. Bayes chains: the posterior after test 1 becomes the prior for test 2.

(Cross-check the chaining view: after test 1, prior $=0.16\overline{6}$. Then $P(D\mid +)=\frac{0.99\cdot 0.16667}{0.99\cdot 0.16667+0.05\cdot 0.83333}=\frac{0.165}{0.165+0.041667}=0.7984$ — same answer.)

Priors: each coin $0.5$. Likelihoods: $P(H\mid \text{fair})=0.5,P(H\mid \text{2-head})=1$. $P(H)=(0.5)(0.5)+(1)(0.5)=0.25+0.5=\mathrm{0.75.}$ $P(\text{2-head}\mid H)=\frac{0.5}{0.75}=\frac{2}{3}\approx \mathrm{0.6667.}$ Next flip — mix the two coins by their posterior weights: $P(H_2\mid H_1)=\underset{\text{2-head}}{\underbrace{\frac{2}{3}}}\cdot 1+\underset{\text{fair}}{\underbrace{\frac{1}{3}}}\cdot \frac{1}{2}=\frac{2}{3}+\frac{1}{6}=\frac{5}{6}\approx \mathrm{0.8333.}$ Seeing one head made you lean toward the two-headed coin, which raises the expected chance of the next head above $0.75$.

Guess likelihood: $P(\text{correct}\mid \text{guess})=\frac{1}{4}=0.25$. $P(\text{correct})=(0.9)(0.7)+(0.25)(0.3)=0.63+0.075=\mathrm{0.705.}$ $P(\text{studied}\mid \text{correct})=\frac{0.63}{0.705}=\frac{0.63}{0.705}\approx \mathrm{0.8936.}$ A correct answer nudges belief in studying from $0.70$ up to $\approx 0.894$ — evidence, but not certainty, because guessers occasionally get lucky.

Priors (colour of first draw): $P(R_1)=\frac{3}{5},P(B_1)=\frac{2}{5}$. Likelihoods for "second is red", now with one ball removed: $P(R_2\mid R_1)=\frac{2}{4}=\frac{1}{2},P(R_2\mid B_1)=\frac{3}{4}.$ Evidence: $P(R_2)=\frac{3}{5}\cdot \frac{1}{2}+\frac{2}{5}\cdot \frac{3}{4}=\frac{3}{10}+\frac{3}{10}=\frac{6}{10}=\frac{3}{5}.$ (Sanity: by symmetry the marginal chance any specific draw is red equals the overall red fraction $3/5$ — reassuring.) Bayes: $P(R_1\mid R_2)=\frac{\frac{3}{10}}{\frac{6}{10}}=\frac{1}{2}.$ Knowing the second ball is red makes the first slightly less likely to have been red than its prior $3/5$ — because a red first draw leaves fewer reds for the second.

Likelihoods of receiving $1$: if $1$ was sent, no flip: $P(r=1\mid s=1)=0.9$; if $0$ was sent, a flip is needed: $P(r=1\mid s=0)=0.1$. $P(r=1)=(0.1)(0.6)+(0.9)(0.4)=0.06+0.36=\mathrm{0.42.}$ $P(s=1\mid r=1)=\frac{0.36}{0.42}=\frac{6}{7}\approx \mathrm{0.8571.}$ Most likely $1$ was sent, posterior $\approx 0.857$. The receiver's optimal decision uses both the prior bit-frequency and the channel reliability — this is the core of Bayesian decoding.

With replacement, draws are independent given the bag, so likelihoods multiply. $P(\text{GGR}\mid A)=0.7\cdot 0.7\cdot 0.3=0.147,P(\text{GGR}\mid B)=0.4\cdot 0.4\cdot 0.6=\mathrm{0.096.}$ Evidence: $P(\text{GGR})=(0.147)(0.5)+(0.096)(0.5)=0.0735+0.048=\mathrm{0.1215.}$ Bayes: $P(A\mid \text{GGR})=\frac{0.0735}{0.1215}\approx \mathrm{0.6049.}$ Bag $A$ is more likely ($\approx 0.605$). Two greens pull toward the greener bag $A$; the single red pulls back a little, but not enough to overturn it.

Every problem above followed the same skeleton — recite it. ::: List priors $P(A_i)$; list matching likelihoods $P(B\mid A_i)$; sum the products to get evidence $P(B)$; divide the one joint you want by $P(B)$. "PLE over E."