Probability Theory & Statistics

Question 1

(a) $Y=-\ln X$, so $X = e^{-Y}$, $x\in(0,1)\Rightarrow y\in(0,\infty)$. $\left|\frac{dx}{dy}\right| = e^{-y}$. (1 for inverse, 1 for Jacobian) $f_Y(y) = f_X(e^{-y})\,e^{-y} = 2e^{-y}\cdot e^{-y} = 2e^{-2y}, \quad y>0.$ (2 for correct density + support)

(b) This is $\text{Exponential}(\lambda=2)$. (2)

(c) MGF of Exp(2): $M_Y(t) = \frac{2}{2-t}$, $t<2$. (1) $M_Y'(t) = \frac{2}{(2-t)^2}$, so $E[Y]=M_Y'(0)=\frac12$. (1) $M_Y''(t) = \frac{4}{(2-t)^3}$, so $E[Y^2] = \frac{4}{8}=\frac12$. (1) $\operatorname{Var}(Y) = \frac12 - \frac14 = \frac14$. (1) (Consistent with Exp(2): mean $1/2$, var $1/4$.)

Question 2

(a) $\int_0^1\int_0^1 c(x+y)\,dx\,dy = c\int_0^1 (\tfrac12 + y)\,dy = c(\tfrac12+\tfrac12)=c=1$. So $c=1$. (2)

(b) $f_X(x)=\int_0^1 (x+y)\,dy = x + \tfrac12$, for $0\le x\le1$. By symmetry $f_Y(y)=y+\tfrac12$. (2) $f_X(x)f_Y(y) = (x+\tfrac12)(y+\tfrac12) \ne x+y$ in general, so not independent. (1)

(c) $f_{Y|X}(y|x) = \dfrac{x+y}{x+\tfrac12}$, $0\le y\le1$. (1) $E[Y|X=x] = \int_0^1 y\,\frac{x+y}{x+\tfrac12}\,dy = \frac{1}{x+\tfrac12}\left(\frac{x}{2}+\frac13\right) = \frac{3x+2}{6x+3}.$ (2)

(d) $E[X]=\int_0^1 x(x+\tfrac12)dx = \tfrac13+\tfrac14=\tfrac{7}{12}=E[Y]$. $E[XY]=\int_0^1\int_0^1 xy(x+y)\,dx\,dy = \int_0^1\int_0^1 (x^2y+xy^2)dxdy = \tfrac13\cdot\tfrac12+\tfrac12\cdot\tfrac13=\tfrac13$. $\operatorname{Cov}(X,Y)=\tfrac13-(\tfrac{7}{12})^2 = \tfrac13-\tfrac{49}{144} = \frac{48-49}{144}=-\frac{1}{144}$. (2)

Question 3

(a) Likelihood $L(\lambda)=\prod \lambda e^{-\lambda t_i}=\lambda^n e^{-\lambda\sum t_i}$. (1) $\ell(\lambda)=n\ln\lambda - \lambda\sum t_i$. (1) $\ell'(\lambda)=\frac{n}{\lambda}-\sum t_i = 0 \Rightarrow \hat\lambda = \frac{n}{\sum t_i} = \frac{1}{\bar t}$. (1) Second derivative $-n/\lambda^2<0$ confirms maximum. (1)

(b) $E[\hat\lambda]=E[1/\bar T] = \frac{n}{n-1}\lambda \ne \lambda$, so biased (overestimates). (1.5) Unbiased estimator: $\tilde\lambda = \frac{n-1}{n}\hat\lambda = \frac{n-1}{\sum t_i}$, since $E[\tilde\lambda] = \frac{n-1}{n}\cdot\frac{n}{n-1}\lambda = \lambda$. (1.5)

(c) $\bar T \xrightarrow{P} 1/\lambda$ by the Weak LLN; $g(x)=1/x$ continuous at $1/\lambda$, so by continuous mapping $\hat\lambda = 1/\bar T \xrightarrow{P}\lambda$. Hence consistent. (3)

Question 4

(a) $\hat p = 220/400 = 0.55$. Under $H_0$, SE $=\sqrt{\frac{0.5\cdot0.5}{400}} = \sqrt{0.000625}=0.025$. (2) $z = \frac{0.55-0.5}{0.025} = 2.0.$ (1) $|z|=2.0 > 1.96$, so reject $H_0$ at 5%. (1) (p-value $=2(1-\Phi(2))=2(0.0228)=0.0456<0.05$.) (1)

(b) CI uses $\hat p$: SE $=\sqrt{\frac{0.55\cdot0.45}{400}}=\sqrt{0.00061875}=0.02487$. $0.55 \pm 1.96(0.02487) = 0.55\pm0.0488 = (0.5012,\,0.5988)$. (2) Interval excludes $0.5$ (just), consistent with rejecting $H_0$. (1)

(c) Type II error: failing to reject $H_0$ (concluding the coin is fair) when it is actually biased. (1) Reduce by increasing sample size $n$ (or accepting larger effect / raising $\alpha$). (1)

Question 5

(a) Likelihood $\propto \theta^3(1-\theta)^7$. Prior $\propto \theta^{1}(1-\theta)^{1}$ (Beta(2,2)). (2) Posterior $\propto \theta^{3+1}(1-\theta)^{7+1} = \theta^{4}(1-\theta)^{8}$, i.e. $\theta \mid \text{data} \sim \text{Beta}(2+3,\,2+7) = \text{Beta}(5,9).$ (2)

(b) Posterior mean $= \frac{5}{5+9}=\frac{5}{14}\approx 0.357$. (2) MLE $=0.3$. Posterior mean is pulled toward the prior mean $0.5$, hence larger than the MLE. (1)

(c) Estimate $\hat p = \frac1N\sum \mathbf 1(\theta_i>0.5)$ is a sample mean of i.i.d. Bernoulli indicators; by LLN it $\to P(\theta>0.5)$ almost surely as $N\to\infty$. (2) Standard error $=\sqrt{p(1-p)/N} = O(N^{-1/2})$. (1)

[
  {"claim":"Q1 Var(Y) for Exp(2) equals 1/4 and E[Y]=1/2 via MGF",
   "code":"t=symbols('t'); M=2/(2-t); E=diff(M,t).subs(t,0); E2=diff(M,t,2).subs(t,0); var=E2-E**2; result=(E==Rational(1,2)) and (var==Rational(1,4))"},
  {"claim":"Q2 Cov(X,Y) = -1/144",
   "code":"x,y=symbols('x y'); f=x+y; EX=integrate(integrate(x*f,(x,0,1)),(y,0,1)); EY=integrate(integrate(y*f,(x,0,1)),(y,0,1)); EXY=integrate(integrate(x*y*f,(x,0,1)),(y,0,1)); cov=EXY-EX*EY; result=(cov==Rational(-1,144))"},
  {"claim":"Q4 z-statistic under H0 equals 2.0",
   "code":"phat=Rational(220,400); se=sqrt(Rational(1,2)*Rational(1,2)/400); z=(phat-Rational(1,2))/se; result=(simplify(z)==2)"},
  {"claim":"Q5 posterior is Beta(5,9) with mean 5/14",
   "code":"a=2+3; b=2+7; mean=Rational(a,a+b); result=(a==5) and (b==9) and (mean==Rational(5,14))"}
]