Exercises — Joint distributions — joint PMF - PDF, marginal, conditional

(a) What we check: both non-negativity and total $=1$ (the two defining conditions). Every entry is $\ge 0$. The sum: $0.05+0.05+0.10+0.10+0.20+0.10+0.10+0.15+0.15=\mathrm{1.00.}✓$ So it is a valid joint PMF.

(b) This is pure reading — the cell in row $X=1$, column $Y=1$: $P(X=1,Y=1)=0.20$.

(c) Why add two cells: the event $\{X=1,Y\le 1\}$ is the union of the disjoint cells $\{X=1,Y=0\}$ and $\{X=1,Y=1\}$. Disjoint $\Rightarrow$ add: $0.10+0.20=\mathrm{0.30.}$

Why no integral is needed: a constant density over a region means "probability spread evenly." Total probability $=$ (height $k$) $\times$ (area of region). The unit square has area $1$, so $k\times 1=1\Rightarrow k=1.$ Sanity picture: a flat sheet of height $1$ sitting on a $1\times 1$ base has volume $1$ — and volume under the density is total probability.

Recipe: $p_X(x)={\sum }_y{p}_{X,Y}(x,y)$ — sum along a row. $p_Y(y)={\sum }_x{p}_{X,Y}(x,y)$ — sum down a column.

Rows (marginal of $X$): $p_X(0)=0.05+0.05+0.10=0.20,p_X(1)=0.10+0.20+0.10=0.40,p_X(2)=0.10+0.15+0.15=\mathrm{0.40.}$ Columns (marginal of $Y$): $p_Y(0)=0.05+0.10+0.10=0.25,p_Y(1)=0.05+0.20+0.15=0.40,p_Y(2)=0.10+0.10+0.15=\mathrm{0.35.}$ Check: $p_X$ sums to $0.20+0.40+0.40=1.00$ and $p_Y$ sums to $0.25+0.40+0.35=1.00$ — both consistent with the total probability being $1$. We summarize in the margins:

So $p_X=(0.20,0.40,0.40)$ and $p_Y=(0.25,0.40,0.35)$. (Writing the sums in the margins is literally where the name "marginal" comes from.)

Recipe: freeze the column $Y=1$, divide each entry by that column's total $p_Y(1)=0.40$. $p_{X\mid Y}(0\mid 1)=\frac{0.05}{0.40}=0.125,p_{X\mid Y}(1\mid 1)=\frac{0.20}{0.40}=0.50,p_{X\mid Y}(2\mid 1)=\frac{0.15}{0.40}=\mathrm{0.375.}$ Why divide: the raw column sums to $p_Y(1)=0.40$, not $1$. Dividing rescales the slice into a legitimate distribution. Check: $0.125+0.50+0.375=1.00✓$.

First confirm it's valid: ${\int }_0^1{\int }_0^1(x+y)dxdy={\int }_0^1\left(\frac{1}{2}+y\right)dy=\frac{1}{2}+\frac{1}{2}=1✓$. Recipe: integrate out $y$ over its full range $0$ to $1$ (a fixed square, so bounds are constant): $f_X(x)={\int }_0^1(x+y)dy=[xy+\frac{y^2}{2}{]}_0^1=x+\frac{1}{2},0<x<1,$ and $f_X(x)=0$ for $x\le 0$ or $x\ge 1$ (outside the support the joint is $0$, so the integral vanishes). Check it integrates to 1: ${\int }_0^1(x+\frac{1}{2})dx=\frac{1}{2}+\frac{1}{2}=1✓$.

(a) Why: total probability must be $1$. On $0<x<y<1$, for a fixed $x$ the variable $y$ runs from $x$ up to $1$ (the orange slice in the figure): ${\int }_0^1{\int }_x^{1}8xydydx={\int }_0^{1}8x[\frac{y^2}{2}{]}_x^{1}dx={\int }_0^{1}4x(1-x^2)dx=4(\frac{1}{2}-\frac{1}{4})=1.✓$

(b) Marginal of $X$ — collapse $y$; for fixed $x$, $y\in (x,1)$: $f_X(x)={\int }_x^{1}8xydy=8x\cdot \frac{1-x^2}{2}=4x(1-x^2),0<x<1,$ and $f_X(x)=0$ outside $(0,1)$. Marginal of $Y$ — collapse $x$; for fixed $y$, $x\in (0,y)$ (a horizontal slice of the triangle): $f_Y(y)={\int }_0^{y}8xydx=8y\cdot \frac{y^2}{2}=4y^3,0<y<1,$ and $f_Y(y)=0$ outside $(0,1)$. Checks: ${\int }_0^{1}4x(1-x^2)dx=1$ and ${\int }_0^{1}4y^{3}dy=1✓$.

(c) Conditional of $Y$ given $X=x$ — divide joint by $f_X(x)$, keep the support $y>x$: $f_{Y\mid X}(y\mid x)=\frac{8xy}{4x(1-x^2)}=\frac{2y}{1-x^2},x<y<1,$ and $f_{Y\mid X}(y\mid x)=0$ for $y\le x$ or $y\ge 1$. Why the support shifts: once $X=x$ is fixed, $Y$ can only live where the joint is positive, i.e. $y\in (x,1)$.

Test: independence $⟺f_{X,Y}(x,y)=f_X(x)f_Y(y)$ for all $(x,y)$. $f_X(x)f_Y(y)=4x(1-x^2)\cdot 4y^3=16x{y}^3(1-x^2),$ but the joint is $8xy$. These are not equal, so $X,Y$ are dependent. Deeper reason (this is the real giveaway): the support itself is the triangle $x<y$ — a constraint linking the two variables. Whenever the support of a joint density is not a rectangle (product of intervals), the variables cannot be independent, no matter how the formula looks. Knowing $X=0.9$ forces $Y>0.9$; that is dependence by definition. See Independence of Random Variables.

(a) Marginals (support $x<y$; for fixed $x$, $y\in (x,1)$; for fixed $y$, $x\in (0,y)$): $f_X(x)={\int }_x^{1}2dy=2(1-x),0<x<1;f_Y(y)={\int }_0^{y}2dx=2y,0<y<1,$ and both marginals are $0$ outside $(0,1)$.

(b) Expectations (use each marginal — see Expectation and Variance): $E[X]={\int }_0^{1}x\cdot 2(1-x)dx=2(\frac{1}{2}-\frac{1}{3})=\frac{1}{3},$ $E[Y]={\int }_0^{1}y\cdot 2ydy=2\cdot \frac{1}{3}=\frac{2}{3}.$

(c) Cross-moment (integrate $xy$ against the joint, over the triangle): $E[XY]={\int }_0^1{\int }_0^{y}xy\cdot 2dxdy={\int }_0^{1}2y\cdot \frac{y^2}{2}dy={\int }_0^1{y}^{3}dy=\frac{1}{4}.$ Then, via Covariance and Correlation, $\mathrm{Cov}(X,Y)=E[XY]-E[X]E[Y]=\frac{1}{4}-\frac{1}{3}\cdot \frac{2}{3}=\frac{1}{4}-\frac{2}{9}=\frac{9-8}{36}=\frac{1}{36}.$

(d) $\mathrm{Cov}=+\frac{1}{36}>0$: $X$ and $Y$ tend to move together. That matches the geometry — since $x<y$ always, a large $x$ forces a large $y$, giving positive association.

Tool: the chain rule $f_{X,Y}(x,y)=f_{X\mid Y}(x\mid y)f_Y(y)$.

• $f_Y(y)=1$ for $0<y<1$.
• Given $Y=y$, $X$ is uniform on $(0,y)$, so $f_{X\mid Y}(x\mid y)=\frac{1}{y}$ for $0<x<y$.

Multiply: $f_{X,Y}(x,y)=\frac{1}{y}\cdot 1=\frac{1}{y},0<x<y<1,$ and $f_{X,Y}=0$ elsewhere. Support: the triangle $0<x<y<1$ (same triangle as before). Recover $f_X$ — collapse $y$; for fixed $x$, $y$ ranges $x$ to $1$: $f_X(x)={\int }_x^1\frac{1}{y}dy=[\ln y{]}_x^1=-\ln x=\ln \frac{1}{x},0<x<1,$ and $f_X(x)=0$ outside $(0,1)$. Check it integrates to 1: ${\int }_0^1(-\ln x)dx=[x-x\ln x{]}_0^1=1✓$. This is exactly the Law of Total Probability carried out in the continuous world.

(a) By symmetry $E[X]=E[Y]=0$ (the density is symmetric about both axes). For $E[XY]$: $E[XY]=\frac{1}{\pi }{\iint }_{x^2+y^2<1}xydxdy.$ For each fixed $x$, $y$ runs symmetrically over $(-\sqrt{1-x^2},\sqrt{1-x^2})$, and ${\int }_{-a}^{a}ydy=0$. So the whole integral is $0$: $E[XY]=0\Rightarrow \mathrm{Cov}(X,Y)=0-0\cdot 0=0.$

(b) Independence test via support: the support is a disk, not a rectangle. Concretely, $f_X(x)=\frac{2\sqrt{1-x^2}}{\pi }$ on $(-1,1)$ (length of the vertical chord over $\pi$) and $0$ elsewhere, and likewise $f_Y(y)=\frac{2\sqrt{1-y^2}}{\pi }$ on $(-1,1)$. Their product $f_X(x)f_Y(y)=\frac{4\sqrt{1-x^2}\sqrt{1-y^2}}{{\pi }^2}$ is not equal to the joint $\frac{1}{\pi }$ on the disk (and is nonzero at, e.g., $(0.9,0.9)$ which lies outside the disk where the joint is $0$). Hence dependent.

(c) The resolution: covariance only detects linear association. Here the dependence is purely geometric/nonlinear: knowing $X=0.9$ squeezes $Y$ into $|y|<\sqrt{1-0.81}=\sqrt{0.19}$, so $X$ constrains the range of $Y$ — but symmetrically, so no linear tilt exists. Zero covariance means "no linear trend," never "independent." Only for the Bivariate Normal Distribution does zero covariance actually force independence. See also Independence of Random Variables.

This is Bayes' Theorem in continuous–discrete mixed form. The conditional PMF of the data given $\theta$ is $P(D=1\mid \Theta =\theta )=\theta$.

Step 1 — joint (chain rule): $f_{\Theta ,D}(\theta ,1)=P(D=1\mid \theta )f_{\Theta }(\theta )=\theta \cdot 1=\theta$ for $0<\theta <1$ (and $0$ elsewhere).

Step 2 — marginal of the data (Law of Total Probability, integrate out $\theta$): $P(D=1)={\int }_0^1\theta d\theta =\frac{1}{2}.$

Step 3 — posterior (Bayes = joint ÷ marginal): $f_{\Theta \mid D}(\theta \mid 1)=\frac{\theta }{1/2}=2\theta ,0<\theta <1,$ and $0$ outside $(0,1)$. Check: ${\int }_0^{1}2\theta d\theta =1✓$ — a valid density.

Step 4 — posterior mean: $E[\Theta \mid D=1]={\int }_0^1\theta \cdot 2\theta d\theta =2\cdot \frac{1}{3}=\frac{2}{3}.$ Interpretation: seeing a defect pushes our belief about $\Theta$ from prior mean $\frac{1}{2}$ up to $\frac{2}{3}$ — exactly what evidence of a defect should do.