Exercises — Joint, marginal, and conditional distributions

WHAT: read the cell in row $X=1$, column $Y=1$. $p_{X,Y}(1,1)=\mathrm{0.12.}$ Meaning: the probability that a random person is both sick and tests positive is $0.12$. This is a raw count-fraction over everyone, not restricted to any subgroup.

WHY margins: the row totals live in the right margin, the column totals in the bottom margin — that is literally where the name comes from.

• Marginals of $X$ = the two row sums: $p_X(0)=0.85,p_X(1)=0.15$.
• Marginals of $Y$ = the two column sums: $p_Y(0)=0.83,p_Y(1)=0.17$.

So $p_X(1)=0.15$ and $p_Y(1)=0.17$.

It is a conditional: joint cell $p_{X,Y}(1,1)=0.12$ divided by the marginal $p_Y(1)=0.17$. By the definition $p_{X\mid Y}(1\mid 1)=\frac{p_{X,Y}(1,1)}{p_Y(1)},$ this is $P(\text{sick}\mid \text{positive})$ — "of the people who tested positive, what fraction are sick?" We slice to the positive column, then divide by its total.

WHAT / WHY: fix $X=1$ (the sick row: $0.03$ and $0.12$). That row sums to $p_X(1)=0.15$, not $1$, so we divide by $0.15$ to re-normalize into a valid distribution. $p_{Y\mid X}(0\mid 1)=\frac{0.03}{0.15}=0.20,p_{Y\mid X}(1\mid 1)=\frac{0.12}{0.15}=\mathrm{0.80.}$ Check: $0.20+0.80=1$. ✓ Among the sick, $80\%$ test positive (the test's sensitivity).

Slice the positive column ($0.05$ and $0.12$), whose total is $p_Y(1)=0.17$, then take the sick fraction: $p_{X\mid Y}(1\mid 1)=\frac{0.12}{0.17}\approx \mathrm{0.7059.}$ So a positive test raises sick-probability from the base rate $0.15$ up to about $0.71$.

Product rule: $p_{X,Y}(x,y)=p_{Y\mid X}(y\mid x)p_X(x)$. $p_{X,Y}(1,1)=p_{Y\mid X}(1\mid 1)\cdot p_X(1)=0.80\times 0.15=\mathrm{0.12.}✓$ This is the inverse move of Exercise 2.1: we sliced-then-divided to get $0.80$; multiplying by the marginal $0.15$ puts the cell back.

Independence requires $p_{X,Y}(x,y)=p_X(x)p_Y(y)$ for every cell. Test the $(1,1)$ cell: $p_X(1)p_Y(1)=0.15\times 0.17=0.0255,p_{X,Y}(1,1)=\mathrm{0.12.}$ $0.0255\ne 0.12$, so the factorization fails on this cell ⇒ dependent. (One failing cell is enough to break independence.) This matches Independence and conditional independence: knowing the test is positive really does change the sick-probability from $0.15$ to $0.71$.

$P(\text{positive}\mid \text{sick})=\frac{p_{X,Y}(1,1)}{p_X(1)}=\frac{0.12}{0.15}=\mathrm{0.80.}$ They differ ($0.706$ vs $0.80$) because the denominators differ: one normalizes over the positive column ($0.17$), the other over the sick row ($0.15$). Swapping the bar swaps which marginal you divide by. To convert one into the other you need Bayes' theorem: $p_{X\mid Y}(1\mid 1)=\frac{p_{Y\mid X}(1\mid 1)p_X(1)}{p_Y(1)}=\frac{0.80\times 0.15}{0.17}\approx \mathrm{0.706.}✓$

The conditional $p_{X\mid Y}(x\mid 0)=\frac{p_{X,Y}(x,0)}{p_Y(0)}$ would be $\frac{0}{0}$ — undefined. That is exactly why the definition carries the condition $p_Y(y)>0$. What it looks like: you are trying to slice a strip that has zero total probability — there is no population to re-normalize, so the question "of the negatives, how many are sick?" is meaningless when there are no negatives. The formula is simply not applicable there.

Let $M\in \{A,B\}$ be the machine, $D$ = defective. Given: $p_M(A)=0.6,p_M(B)=0.4$, $p_{D\mid M}(1\mid A)=0.02,p_{D\mid M}(1\mid B)=0.05$. Marginalize $M$ away (this is Marginalization and the law of total probability): $P(D)={\sum }_m{p}_{D\mid M}(1\mid m)p_M(m)=0.02\cdot 0.6+0.05\cdot 0.4=0.012+0.020=\mathrm{0.032.}$ Then Bayes to reverse the bar: $P(A\mid D)=\frac{p_{D\mid M}(1\mid A)p_M(A)}{P(D)}=\frac{0.012}{0.032}=\mathrm{0.375.}$ So even though $A$ makes most parts, only $37.5\%$ of defects come from it — because $A$ is the cleaner machine.

(a) Valid? A pdf must be $\ge 0$ (it is, on the square) and integrate to $1$: ${\int }_0^1{\int }_0^1(x+y)dxdy={\int }_0^1\left(\frac{1}{2}+y\right)dy=\frac{1}{2}+\frac{1}{2}=1.✓$ (b) Marginal of $X$ — integrate $y$ away: $f_X(x)={\int }_0^1(x+y)dy=x+\frac{1}{2},0\le x\le 1.$ (c) Conditional of $Y$ given $X=x$ — slice then divide: $f_{Y\mid X}(y\mid x)=\frac{x+y}{x+\frac{1}{2}},0\le y\le 1.$ (d) Independent? The conditional still depends on $x$, so knowing $x$ changes the shape of $Y$'s distribution ⇒ dependent. Equivalently $f_X(x)f_Y(y)=(x+\frac{1}{2})(y+\frac{1}{2})\ne x+y$. Contrast with the parent's $4xy$ example, which did factor.

Conditional independence given $X$ (see Naive Bayes classifier) means the joint feature likelihood factors: $p(Y_1,Y_2\mid X)=p(Y_1\mid X)p(Y_2\mid X).$ Unnormalized scores (numerator of Bayes = likelihood × prior): $\text{spam: }0.8\times 0.6\times 0.3=0.144,\text{ham: }0.1\times 0.2\times 0.7=\mathrm{0.014.}$ Normalize (the denominator is their sum — marginalizing the label away): $P(\text{spam}\mid Y_1=1,Y_2=1)=\frac{0.144}{0.144+0.014}=\frac{0.144}{0.158}\approx \mathrm{0.9114.}$ The two "free-ish" words together push spam-probability from a prior $0.3$ up to about $0.91$.

Let $G={\sum }_{x}g(x)$ and $H={\sum }_{y}h(y)$. Since the joint sums to $1$: $1={\sum }_x{\sum }_{y}g(x)h(y)=({\sum }_{x}g(x))({\sum }_{y}h(y))=GH.$ Marginals: $p_X(x)={\sum }_{y}g(x)h(y)=g(x)H,p_Y(y)={\sum }_{x}g(x)h(y)=h(y)G.$ Product of marginals: $p_X(x)p_Y(y)=g(x)H\cdot h(y)G=g(x)h(y)(GH)=g(x)h(y)\cdot 1=p_{X,Y}(x,y).$ So the joint factors into the marginals ⇒ independent. $■$ The lesson: independence only needs the joint to split into a product of a pure-$x$ piece and a pure-$y$ piece — the normalizing constants sort themselves out. (This is why $4xy$ was independent but $x+y$ was not: $x+y$ can't be written $g(x)h(y)$.)