Exercises — Random variables (discrete and continuous)

Type. The outcomes are $\{1,2,3,4,5,6\}$ — a finite countable set. So $X$ is discrete.

Probability. A fair die means all six faces are equally likely, so each has probability $\frac{1}{6}$. Therefore $P(X=4)=\frac{1}{6}\approx \mathrm{0.1667.}$ Check the PMF is valid: ${\sum }_{x=1}^6\frac{1}{6}=6\cdot \frac{1}{6}=1$ ✓ (normalization holds).

False. For a continuous random variable, the probability of any single exact value is $0$: $P(T=5)={\int }_5^{5}f(t)dt=0.$ The function $f(5)$ is a density (probability per unit time), not a probability. To get a real probability you must integrate over an interval, e.g. $P(4.9\le T\le 5.1)={\int }_{4.9}^{5.1}f(t)dt$.

Using $E[X]={\sum }_{x}xP(X=x)$: $E[X]=0\cdot (1-0.3)+1\cdot (0.3)=\mathrm{0.3.}$ This matches the parent note's result $E[X]=p$. See Bernoulli and Categorical Distributions.

PMF. $P(X=k)=\left(\genfrac{}{}{0ex}{}{n}{k}\right)p^k(1-p{)}^{n-k}$. With $n=10$, $k=2$, $p=0.3$: $\left(\genfrac{}{}{0ex}{}{10}{2}\right)=\frac{10!}{2!8!}=45,$ $P(X=2)=45\cdot (0.3{)}^2\cdot (0.7{)}^8=45\cdot 0.09\cdot 0.05764801\approx \mathrm{0.2335.}$ Mean. Each flip contributes $p$, and expectation adds over independent parts: $E[X]=np=10\cdot 0.3=3.$

Density. On $[a,b]=[2,8]$, $f(x)=\frac{1}{b-a}=\frac{1}{6}$ for every $x$ in the interval, so $f(5)=\frac{1}{6}\approx 0.1667$. Probability = area. The region under a flat line is a rectangle of height $\frac{1}{6}$ and width $6-3=3$: $P(3\le X\le 6)={\int }_3^6\frac{1}{6}dx=\frac{1}{6}\cdot 3=\frac{1}{2}.$

Definition. $\text{Var}(X)=E[X^2]-(E[X]{)}^2$ (see Expectation and Variance). Compute $E[X^2]$: $E[X^2]={\int }_a^b{x}^2\cdot \frac{1}{b-a}dx=\frac{1}{b-a}\cdot \frac{x^3}{3}{|}_a^b=\frac{b^3-a^3}{3(b-a)}.$ Factor $b^3-a^3=(b-a)(b^2+ab+a^2)$: $E[X^2]=\frac{b^2+ab+a^2}{3}.$ Subtract $(E[X]{)}^2={\left(\frac{a+b}{2}\right)}^2=\frac{a^2+2ab+b^2}{4}$: $\text{Var}(X)=\frac{b^2+ab+a^2}{3}-\frac{a^2+2ab+b^2}{4}=\frac{4(b^2+ab+a^2)-3(a^2+2ab+b^2)}{12}=\frac{a^2-2ab+b^2}{12}=\frac{(b-a{)}^2}{12}.$ Evaluate for $[2,8]$: $\frac{(8-2{)}^2}{12}=\frac{36}{12}=3.$

Linearity of expectation: $E[Y]=E[2X+1]=2E[X]+1=2(0.3)+1=\mathrm{1.6.}$ Variance under linear scaling: adding a constant shifts values but never changes spread, and multiplying by $2$ stretches deviations by $2$, so variance scales by $2^2=4$: $\text{Var}(Y)=4\text{Var}(X).$ For Bernoulli, $\text{Var}(X)=p(1-p)=0.3\cdot 0.7=0.21$, so $\text{Var}(Y)=4\cdot 0.21=\mathrm{0.84.}$

(a) Normalize. A valid PDF integrates to $1$: ${\int }_0^{2}cxdx=c\cdot \frac{x^2}{2}{|}_0^2=c\cdot \frac{4}{2}=2c=1\Rightarrow c=\frac{1}{2}.$ (b) Expected value. $E[X]={\int }_0^{2}x\cdot \frac{1}{2}xdx=\frac{1}{2}{\int }_0^2{x}^{2}dx=\frac{1}{2}\cdot \frac{x^3}{3}{|}_0^2=\frac{1}{2}\cdot \frac{8}{3}=\frac{4}{3}\approx \mathrm{1.333.}$ The mean sits past the midpoint ($1$) because density grows toward the right — more probability mass lives at larger $x$. (c) CDF-style probability. $P(X\le 1)={\int }_0^1\frac{1}{2}xdx=\frac{1}{2}\cdot \frac{x^2}{2}{|}_0^1=\frac{1}{2}\cdot \frac{1}{2}=\frac{1}{4}.$

Likelihood. For Bernoulli data, $L(p)={\prod }_i{p}^{x_i}(1-p{)}^{1-x_i}$. With all five $x_i=1$: $L(p)=p^5.$ Maximize. Take the log (monotone, easier): $\ell (p)=5\ln p$. Differentiate and set to zero: $\frac{d\ell }{dp}=\frac{5}{p}=0.$ This has no interior solution; $\ell$ is increasing on $(0,1)$, so the maximum is at the boundary $\hat{p}=1$. In general $\hat{p}=\frac{\text{successes}}{n}=\frac{5}{5}=1$. Warning. $\hat{p}=1$ claims failure is impossible — obviously overconfident from only 5 flips. This is why we regularize / use priors in practice.

Mean of $Z$. By linearity, $E[Z]=\frac{1}{\sigma }(E[X]-\mu )=\frac{1}{\sigma }(\mu -\mu )=0.$ Variance of $Z$. Subtracting a constant $\mu$ doesn't change spread; dividing by $\sigma$ scales variance by $\frac{1}{{\sigma }^2}$: $\text{Var}(Z)=\frac{1}{{\sigma }^2}\text{Var}(X)=\frac{1}{{\sigma }^2}\cdot {\sigma }^2=1.$ So $Z\sim \mathcal{N}(0,1)$, the standard normal. Interval translation. The event $-1\le Z\le 1$ is exactly the event $\mu -\sigma \le X\le \mu +\sigma$ (multiply through by $\sigma$, add $\mu$). Since the linear map is a bijection, probabilities are preserved: $P(\mu -\sigma \le X\le \mu +\sigma )=P(-1\le Z\le 1)\approx \mathrm{0.6827.}$ This is the famous "68% within one standard deviation" rule.

Mean. Linearity of expectation: $E[\bar{X}]=\frac{1}{n}{\sum }_{i=1}^{n}E[X_i]=\frac{1}{n}\cdot n\mu =\mu .$ The sample mean is unbiased. Variance. For independent variables, variances add, and the $\frac{1}{n}$ factor pulls out squared: $\text{Var}(\bar{X})=\frac{1}{n^2}{\sum }_{i=1}^n\text{Var}(X_i)=\frac{1}{n^2}\cdot n{\sigma }^2=\frac{{\sigma }^2}{n}.$ Evaluate. $\frac{{\sigma }^2}{n}=\frac{4}{25}=0.16$, so the standard error is $\sqrt{0.16}=0.4$. CLT link. As $n$ grows, $\text{Var}(\bar{X})\to 0$: averages concentrate around $\mu$. The CLT sharpens this — $\bar{X}$ becomes approximately $\mathcal{N}(\mu ,{\sigma }^2/n)$ regardless of the original distribution's shape.