Probability Theory & Statistics

Question 1 (10)

(a) Definition: $M_X(t)=E[e^{tX}]$ (for $t$ in a neighbourhood of 0 where the expectation is finite). (1) Assuming we may interchange differentiation and expectation (valid because $M_X$ exists on an open interval about 0): $M_X'(t)=E[Xe^{tX}]\Rightarrow M_X'(0)=E[X]$. (1.5) $M_X''(t)=E[X^2e^{tX}]\Rightarrow M_X''(0)=E[X^2]$. (1.5)

(b) $M_X(t)=\sum_{k=0}^\infty e^{tk}\frac{e^{-\lambda}\lambda^k}{k!}=e^{-\lambda}\sum_{k=0}^\infty\frac{(\lambda e^t)^k}{k!}=e^{-\lambda}e^{\lambda e^t}=e^{\lambda(e^t-1)}$. (3)

(c) $M_X'(t)=\lambda e^t e^{\lambda(e^t-1)}$, so $E[X]=M_X'(0)=\lambda$. (1) $M_X''(t)=(\lambda e^t+\lambda^2 e^{2t})e^{\lambda(e^t-1)}$, $M_X''(0)=\lambda+\lambda^2=E[X^2]$. (1) $\operatorname{Var}(X)=\lambda+\lambda^2-\lambda^2=\lambda$. (1)

Question 2 (10)

(a) For $x>0$: $F_X(x)=P(X\le x)=P(-\frac1\lambda\ln U\le x)=P(\ln U\ge -\lambda x)=P(U\ge e^{-\lambda x})=1-e^{-\lambda x}$ (since $U$ uniform). (3) Differentiate: $f_X(x)=\lambda e^{-\lambda x}$, $x>0$ — Exponential($\lambda$). (2)

(b) Since $F^{-1}(u)=-\frac1\lambda\ln(1-u)$ (or $-\frac1\lambda\ln u$ as $1-U\sim U$), applying the inverse CDF to a uniform draw yields an exponential draw — the inverse-transform / inverse-CDF sampling method. (2)

(c)

import numpy as np
def mc_mean(n, lam):
    U = np.random.rand(n)
    X = -np.log(U)/lam
    return X.mean()

Convergence to $E[X]=1/\lambda$ guaranteed by the (Strong) Law of Large Numbers. (3)

Question 3 (12)

(a) Likelihood $L=\prod\lambda e^{-\lambda x_i}=\lambda^n e^{-\lambda\sum x_i}$; log-likelihood $\ell=n\ln\lambda-\lambda\sum x_i$. (2) $\frac{d\ell}{d\lambda}=\frac n\lambda-\sum x_i=0\Rightarrow \hat\lambda_{MLE}=\frac{n}{\sum x_i}=\frac1{\bar X}$. Second derivative $-n/\lambda^2<0$ ⇒ max. (2)

(b) $E[X]=1/\lambda$; set $\bar X=1/\lambda\Rightarrow\hat\lambda_{MM}=1/\bar X$. Same as MLE. (2)

(c) $E[\hat\lambda_{MLE}]=E[1/\bar X]=\frac{n}{n-1}\lambda\neq\lambda$. So it is biased (overestimates), bias $=\frac{\lambda}{n-1}$. (3)

(d) Consistency: $\hat\lambda_n\xrightarrow{P}\lambda$ as $n\to\infty$. By LLN $\bar X\xrightarrow{P}1/\lambda$, and by continuous mapping $1/\bar X\xrightarrow{P}\lambda$. Also bias $\to0$ and variance $\to0$. (3)

Question 4 (10)

(a) If $X_1,\dots,X_n$ i.i.d. with mean $\mu$, variance $\sigma^2\in(0,\infty)$, then $Z_n=\frac{\sum X_i-n\mu}{\sigma\sqrt n}=\frac{\sqrt n(\bar X-\mu)}{\sigma}\xrightarrow{d} N(0,1).$ (3)

(b) Let $Y_i=(X_i-\mu)/\sigma$, mean 0, var 1, MGF $m(t)$ with $m(0)=1,m'(0)=0,m''(0)=1$. Then $Z_n=\frac1{\sqrt n}\sum Y_i$ and $M_{Z_n}(t)=\left[m\!\left(\tfrac{t}{\sqrt n}\right)\right]^n.$ (2) Taylor: $m(t/\sqrt n)=1+\frac{t^2}{2n}+o(1/n)$. (1.5) So $M_{Z_n}(t)=\left(1+\frac{t^2}{2n}+o(1/n)\right)^n\to e^{t^2/2}$, the MGF of $N(0,1)$; convergence of MGF ⇒ convergence in distribution. (1.5)

(c) SE of $\bar X=\sqrt{(35/12)/100}=\sqrt{0.029167}=0.17078$. $z=(3.7-3.5)/0.17078=1.1711$. $P(\bar X>3.7)\approx1-\Phi(1.171)\approx0.121$. (2)

Question 5 (10)

(a) Pivot $Z=\frac{\bar X-\mu}{\sigma/\sqrt n}\approx N(0,1)$. CI: $\bar x\pm z_{0.025}\frac{\sigma}{\sqrt n}$. $\frac{\sigma}{\sqrt n}=90/\sqrt{40}=14.230$; margin $=1.96\times14.230=27.89$. CI $=1180\pm27.89=(1152.1,\ 1207.9)$. (4)

(b) $z=\frac{1180-1200}{14.230}=-1.4055$. $|z|<1.96$ ⇒ fail to reject $H_0$. Two-sided p-value $=2(1-\Phi(1.4055))\approx2(0.0799)=0.160$. (4)

(c) Type I: rejecting $H_0$ (concluding mean ≠ 1200) when in fact $\mu=1200$. Type II: failing to reject $H_0$ when the true mean differs from 1200. (2)

Question 6 (8)

(a) $f_X(x)=\int_0^1(x+y)\,dy=x+\frac12$, $0<x<1$. (2)

(b) $E[X]=\int_0^1 x(x+\frac12)dx=\frac13+\frac14=\frac{7}{12}$. (1.5) $E[XY]=\int_0^1\int_0^1 xy(x+y)\,dx\,dy=\int\int(x^2y+xy^2)=\frac13\cdot\frac12+\frac12\cdot\frac13=\frac16+\frac16=\frac13$. (1.5)

(c) By symmetry $E[Y]=7/12$. $\operatorname{Cov}=\frac13-(\frac{7}{12})^2=\frac13-\frac{49}{144}=\frac{48-49}{144}=-\frac1{144}$. Nonzero ⇒ not independent (also $f\neq f_Xf_Y$). (3)

[
{"claim":"Poisson MGF gives mean and variance lambda","code":"t,lam=symbols('t lambda',positive=True); M=exp(lam*(exp(t)-1)); mean=diff(M,t).subs(t,0); m2=diff(M,t,2).subs(t,0); var=simplify(m2-mean**2); result=(simplify(mean-lam)==0 and simplify(var-lam)==0)"},
{"claim":"X=-ln(U)/lam has exponential pdf; E[X]=1/lam","code":"lam,x=symbols('lambda x',positive=True); f=lam*exp(-lam*x); EX=integrate(x*f,(x,0,oo)); result=simplify(EX-1/lam)==0"},
{"claim":"MLE E[1/Xbar]=n/(n-1)*lam so biased","code":"n,lam=symbols('n lambda',positive=True); bias=n/(n-1)*lam-lam; result=simplify(bias-lam/(n-1))==0"},
{"claim":"Cov(X,Y)=-1/144 for f=x+y","code":"x,y=symbols('x y'); f=x+y; EX=integrate(integrate(x*f,(y,0,1)),(x,0,1)); EY=integrate(integrate(y*f,(y,0,1)),(x,0,1)); EXY=integrate(integrate(x*y*f,(y,0,1)),(x,0,1)); cov=simplify(EXY-EX*EY); result=cov==Rational(-1,144)"},
{"claim":"Die CI/z-test: z for Q5b approx -1.4055","code":"z=(1180-1200)/(90/sqrt(40)); result=abs(float(z)+1.40554)<0.01"}
]