Exercises — Probability mass and density functions

A valid PMF must satisfy two things, both from the parent definition:

1. Every height is between $0$ and $1$: here $0.2,0.5,0.3$ all lie in $[0,1]$. ✓
2. The heights sum to exactly $1$: $0.2+0.5+0.3=1$. ✓

Both hold, so yes, $g$ is a valid PMF. Notice we only added the values — for a discrete variable each height already is a probability, so summing them is the whole test.

They are wrong. $h$ is a candidate PDF, and a PDF is a density, not a probability — it is allowed to exceed $1$. The only tests are:

1. $h(x)\ge 0$ everywhere: $2\ge 0$ and $0\ge 0$. ✓
2. Total area equals $1$: ${\int }_0^{0.5}2dx=2\times 0.5=1.✓$ The height being $2$ is fine because the region is only $0.5$ wide; height $\times$ width $=1$. Look at figure below: a tall-but-narrow rectangle can still enclose area $1$.

Each ordered pair $(i,j)$ has probability $\frac{1}{36}$. $P(X=5)$: pairs summing to $5$ are $(1,4),(2,3),(3,2),(4,1)$ — that is $4$ outcomes. $p_X(5)=\frac{4}{36}=\frac{1}{9}.$ $P(X\le 4)$: add masses for $x=2,3,4$. Counts: sum $2$ has $1$ way, sum $3$ has $2$ ways, sum $4$ has $3$ ways. $P(X\le 4)=\frac{1+2+3}{36}=\frac{6}{36}=\frac{1}{6}.$ We summed masses — legal because the events $\{X=2\},\{X=3\},\{X=4\}$ are disjoint (a single roll gives one sum), so their probabilities add.

A PDF gives probability by area (integration): $P(1\le X\le 3)={\int }_1^3\frac{1}{4}dx=\frac{1}{4}(3-1)=\frac{1}{4}\times 2=\mathrm{0.5.}$ For a single point: $P(X=2)={\int }_2^2\frac{1}{4}dx=0,$ because the strip from $2$ to $2$ has zero width, hence zero area. This is the "points have zero probability" property in action.

Normalization forces the total area to be $1$. Why this tool (an integral)? Because for a density, "total probability" is the area under the whole curve: ${\int }_0^{2}cxdx=c{\left[\frac{x^2}{2}\right]}_0^2=c\cdot \frac{4}{2}=2c.$ Setting $2c=1$ gives $c=\frac{1}{2}$. Check $f_X\ge 0$: $\frac{1}{2}x\ge 0$ on $[0,2]$. ✓ (A rising triangle — see figure.)

$P(X>1)$: integrate from $1$ to $2$: $P(X>1)={\int }_1^2\frac{1}{2}xdx=\frac{1}{2}{\left[\frac{x^2}{2}\right]}_1^2=\frac{1}{2}\cdot \frac{4-1}{2}=\frac{1}{2}\cdot \frac{3}{2}=\frac{3}{4}.$ More area sits on the right because the density grows with $x$ — bigger values are more likely here.

The CDF already is the accumulated area from $0$ to $x$, so no new integral is needed: $P(X\le 1)=F_X(1)=1-e^{-2\times 1}=1-e^{-2}\approx 1-0.1353=\mathrm{0.8647.}$ Interpretation: with rate $\lambda =2$, roughly $86.5\%$ of waits are under one time unit — the decay $e^{-\lambda x}$ makes long waits rare.

Expectation weights each value by its mass: $E[X]=0\cdot (1-p)+1\cdot p=p.$ With $p=0.3$: $E[X]=0.3$. The $x=0$ term vanishes (multiplied by $0$), so the mean of a Bernoulli is simply its success probability. This is why in maximum likelihood the fitted $\hat{p}$ turns out to be the observed fraction of successes.

The CDF is accumulated area ${\int }_{-\infty }^x{f}_X(t)dt$. We must cover every case:

• $x<0$: no density yet, area $=0$. So $F_X(x)=0$.
• $0\le x\le 4$: area of a rectangle of height $\frac{1}{4}$ and width $x$: $F_X(x)={\int }_0^x\frac{1}{4}dt=\frac{x}{4}$.
• $x>4$: all probability already accumulated: $F_X(x)=1$. $F_X(x)=\{\begin{array}{ll}0& x<0\\ \frac{x}{4}& 0\le x\le 4\\ 1& x>4\end{array}$ Differentiating the middle piece returns $\frac{d}{dx}\frac{x}{4}=\frac{1}{4}=f_X(x)$, confirming the PDF–CDF link. See the ramp figure.

Exponentiate each logit (this is what makes every value positive): $e^{z_1}=e^2\approx 7.389,e^{z_2}=e^0=1.$ Denominator: $7.389+1=8.389$. Then $p_1=\frac{7.389}{8.389}\approx 0.8808,p_2=\frac{1}{8.389}\approx \mathrm{0.1192.}$ Validity: each $p_k>0$ and $p_1+p_2=1$ by construction (the denominator is the sum of numerators). Softmax is literally a machine for turning arbitrary scores into a legal PMF over classes — the discrete cousin of everything in this note.

(a) Normalize. Total area must be $1$: ${\int }_{-1}^{1}k(1-x^2)dx=k{\left[x-\frac{x^3}{3}\right]}_{-1}^1=k\left[\left(1-\frac{1}{3}\right)-\left(-1+\frac{1}{3}\right)\right]=k\cdot \frac{4}{3}.$ Set $\frac{4}{3}k=1\Rightarrow k=\frac{3}{4}$. Check $f_X\ge 0$: $1-x^2\ge 0$ on $[-1,1]$. ✓ (A dome — see figure.)

(b) Symmetry argument. The density is even ($f_X(-x)=f_X(x)$) and centered at $0$, so probability mass balances around $0$; the mean must be $0$. Confirm: $E[X]={\int }_{-1}^{1}x\cdot \frac{3}{4}(1-x^2)dx=\frac{3}{4}{\int }_{-1}^1(x-x^3)dx.$ The integrand $x-x^3$ is odd, so its integral over a symmetric interval $[-1,1]$ is $0$. Hence $E[X]=0$, matching the symmetry argument.

A PMF value is itself a probability so it must lie in $[0,1]$. A PDF value is a density (probability per unit length) so it may exceed $1$. For a continuous variable, $P(X=c)$ equals ::: $0$, because the integrating strip has zero width. To get a probability from a PDF you must ::: integrate over an interval (take the area). The first step whenever a PDF has an unknown constant is ::: set the total integral equal to 1 (normalize).