Exercises — Moment generating function (MGF) — definition, use

Match to a known form. In the parent note we derived the Bernoulli MGF $M(t)=(1-p)+p{e}^t$. This is exactly that shape, so $X\sim$ ==Bernoulli$(p)$==. Get the mean. Differentiate: $M_X^{\prime }(t)=p{e}^t$, so $M_X^{\prime }(0)=p{e}^0=p$. Thus $E[X]=p$. Sanity check: $M_X(0)=1-p+p=1$. ✓ (Always $1$.)

This is the Exponential$(\lambda )$ MGF (parent Example 1). It exists only when the exponent inside the defining integral stays negative, i.e. $t<\lambda$; at $t=\lambda$ the denominator hits $0$ and the value blows up, and for $t>\lambda$ the integral diverges — no MGF there. Mean: $M_X^{\prime }(t)=\frac{\lambda }{(\lambda -t{)}^2}$, so $E[X]=M_X^{\prime }(0)=\frac{1}{\lambda }$. With $\lambda =2$, $E[X]=\frac{1}{2}$.

The Normal MGF is $\exp (\mu t+\frac{1}{2}{\sigma }^2{t}^2)$ (parent Example 3). Match term by term:

• Coefficient of $t$: $\mu =3$.
• Coefficient of $t^2$: $\frac{1}{2}{\sigma }^2=2\Rightarrow {\sigma }^2=4$.

So $X\sim N(3,4)$, i.e. mean $3$, variance $4$, standard deviation $2$.

Why derivatives? $E[X^n]=M_X^{(n)}(0)$: each derivative "peels off" one more moment. Write $M_X(t)=\lambda (\lambda -t{)}^{-1}$ and differentiate, using $\frac{d}{dt}(\lambda -t{)}^{-k}=k(\lambda -t{)}^{-k-1}$:

• $M_X^{\prime }(t)=\lambda (\lambda -t{)}^{-2}$
• $M_X^{\prime \prime }(t)=2\lambda (\lambda -t{)}^{-3}$
• $M_X^{\prime \prime \prime }(t)=6\lambda (\lambda -t{)}^{-4}$

Set $t=0$: $E[X^3]=M_X^{\prime \prime \prime }(0)=6\lambda \cdot {\lambda }^{-4}=\frac{6}{{\lambda }^3}$. (General pattern: $E[X^n]=\frac{n!}{{\lambda }^n}$.)

Set up the sum. By definition $M_X(t)={\sum }_{k=0}^{\infty }{e}^{tk}{e}^{-\lambda }\frac{{\lambda }^k}{k!}$. Why regroup? Pull the constant $e^{-\lambda }$ out and merge $e^{tk}{\lambda }^k=(\lambda e^t{)}^k$: $M_X(t)=e^{-\lambda }{\sum }_{k=0}^{\infty }\frac{(\lambda e^t{)}^k}{k!}.$ Recognize the exponential series. ${\sum }_{k\ge 0}\frac{u^k}{k!}=e^u$ with $u=\lambda e^t$: $M_X(t)=e^{-\lambda }{e}^{\lambda e^t}=\exp (\lambda (e^t-1)).$ Mean. $M_X^{\prime }(t)=\lambda e^t\exp (\lambda (e^t-1))$, so $M_X^{\prime }(0)=\lambda \cdot 1\cdot e^0=\lambda$. Thus $E[X]=\lambda$. Check $M_X(0)=\exp (\lambda (1-1))=e^0=1$. ✓

$M_X(t)=1-p+p{e}^t$, so $M_X^{\prime }(t)=p{e}^t$ and $M_X^{\prime \prime }(t)=p{e}^t$.

• $E[X]=M_X^{\prime }(0)=p$.
• $E[X^2]=M_X^{\prime \prime }(0)=p$.

$\mathrm{Var}(X)=E[X^2]-(E[X]{)}^2=p-p^2=p(1-p)$.

Use shift & scale with $b=0$: $M_Y(t)=M_X(at)=\frac{\lambda }{\lambda -at}$. Rewrite to recognize it. Divide top and bottom by $a$: $\frac{\lambda /a}{\lambda /a-t}$. Set $\mu =\lambda /a$: $M_Y(t)=\frac{\mu }{\mu -t},\mu =\frac{\lambda }{a}.$ That is the Exponential MGF with rate $\mu =\lambda /a$, so $Y\sim$ Exponential$(\lambda /a)$. Domain. The original needs $at<\lambda$, i.e. $t<\lambda /a=\mu$ — matching the new rate's domain, as it must.

$M_X(t)=\exp (\mu t+\frac{1}{2}{\sigma }^2{t}^2)$. Apply shift & scale ($a=c,b=d$): $M_Y(t)=e^{dt}{M}_X(ct)=e^{dt}\exp (\mu (ct)+\frac{1}{2}{\sigma }^2(ct{)}^2).$ Combine exponents (add what's in the exponent): $M_Y(t)=\exp ((c\mu +d)t+\frac{1}{2}(c^2{\sigma }^2)t^2).$ This is the Normal MGF with new mean $c\mu +d$ and new variance $c^2{\sigma }^2$. By uniqueness, $Y\sim N(c\mu +d,c^2{\sigma }^2)$. A linear map of a Normal is Normal.

Test finiteness near $0$. For any $t>0$, look at the right tail of $M_X(t)={\int }_{-\infty }^{\infty }{e}^{tx}\frac{1}{\pi (1+x^2)}dx$. As $x\to +\infty$, $e^{tx}$ grows exponentially while $\frac{1}{\pi (1+x^2)}$ only decays like $1/x^2$. Exponential growth crushes polynomial decay, so the integrand $\to \infty$ and the integral diverges for every $t>0$. By symmetry $t<0$ fails on the left tail. So the only $t$ where the integral is finite is $t=0$ — but the MGF must be finite on an open interval around $0$. Therefore the Cauchy has no MGF; you must switch to the Characteristic Function $E[e^{itX}]$, whose $|e^{itx}|=1$ keeps the integral finite always.

Sum of independents = product of MGFs. Each factor is $\frac{\lambda }{\lambda -t}$ (for $t<\lambda$): $M_S(t)={\prod }_{i=1}^n\frac{\lambda }{\lambda -t}={\left(\frac{\lambda }{\lambda -t}\right)}^n,t<\lambda .$ Because all $n$ factors are identical, the product is just the $n$-th power. This is the Gamma$(n,\lambda )$ MGF — obtained without ever integrating the Gamma density. Mean via the shortcut. $\ln M_S(t)=n\ln \lambda -n\ln (\lambda -t)$, so $\frac{d}{dt}\ln M_S(t)=\frac{n}{\lambda -t}$, giving $E[S]=n/\lambda$ at $t=0$ — as expected from $n$ copies each of mean $1/\lambda$.

From L2.2, $M_X(t)=\exp ({\lambda }_1(e^t-1))$ and $M_Y(t)=\exp ({\lambda }_2(e^t-1))$. Multiply (independence): $M_{X+Y}(t)=\exp ({\lambda }_1(e^t-1))\exp ({\lambda }_2(e^t-1))=\exp (({\lambda }_1+{\lambda }_2)(e^t-1)).$ That is precisely the Poisson MGF with rate ${\lambda }_1+{\lambda }_2$. By uniqueness, $X+Y\sim$ Poisson$({\lambda }_1+{\lambda }_2)$. Poisson rates simply add.

Multiply the two Normal MGFs (independence): $M_{X+Y}(t)=\exp ({\mu }_{1}t+\frac{1}{2}{\sigma }_1^2{t}^2)\exp ({\mu }_{2}t+\frac{1}{2}{\sigma }_2^2{t}^2).$ Add exponents: $M_{X+Y}(t)=\exp (({\mu }_1+{\mu }_2)t+\frac{1}{2}({\sigma }_1^2+{\sigma }_2^2)t^2).$ This is the Normal MGF with mean ${\mu }_1+{\mu }_2$ and variance ${\sigma }_1^2+{\sigma }_2^2$. So $X+Y\sim N({\mu }_1+{\mu }_2,{\sigma }_1^2+{\sigma }_2^2)$. Means add and (for independents) variances add — never standard deviations.

Scale first. Each $X_i/\sqrt{n}$ has MGF $M(t/\sqrt{n})$ (shift & scale, $a=1/\sqrt{n}$). By independence the product over $n$ copies gives $M_{S_n}(t)=\left[M\right(t/\sqrt{n}){]}^n.$ Taylor-expand $M$ near $0$ (parent note: $M(s)=1+E[X]s+\frac{1}{2}E[X^2]s^2+\cdots$). With $E[X]=0,E[X^2]=1$ and $s=t/\sqrt{n}$: $M\left(\frac{t}{\sqrt{n}}\right)=1+0\cdot \frac{t}{\sqrt{n}}+\frac{1}{2}\cdot \frac{t^2}{n}+o\left(\frac{1}{n}\right)=1+\frac{t^2}{2n}+o\left(\frac{1}{n}\right).$ Take the $n$-th power and the limit. Using $(1+\frac{a}{n}+o(1/n){)}^n\to e^a$ with $a=\frac{t^2}{2}$: $M_{S_n}(t)\to e^{t^2/2}.$ This is the standard Normal MGF. By uniqueness, $S_n$ converges in distribution to $N(0,1)$ — the Central Limit Theorem, driven entirely by the $\frac{1}{2}{s}^2$ term surviving the $\sqrt{n}$ scaling.

Why take a log? The chain rule on $\ln$ turns the messy derivatives of $M$ into clean central quantities. First derivative: $K_X^{\prime }(t)=\frac{M_X^{\prime }(t)}{M_X(t)}$. At $t=0$, $M_X(0)=1$ and $M_X^{\prime }(0)=E[X]$, so $K_X^{\prime }(0)=\frac{E[X]}{1}=E[X].$ Second derivative (quotient rule): $K_X^{\prime \prime }(t)=\frac{M_X^{\prime \prime }(t)M_X(t)-(M_X^{\prime }(t){)}^2}{M_X(t{)}^2}.$ At $t=0$: numerator $=E[X^2]\cdot 1-(E[X]{)}^2$, denominator $=1$, so $K_X^{\prime \prime }(0)=E[X^2]-(E[X]{)}^2=\mathrm{Var}(X).$ So the log-MGF's first two derivatives at $0$ hand you mean and variance directly — cleaner than raw moments.

Read the definition backwards. For a discrete $X$, $M_X(t)={\sum }_x{e}^{tx}P(X=x)$. So each term $c{e}^{tx}$ says "the value $x$ has probability $c$." Matching:

• $\frac{1}{4}{e}^{0t}\Rightarrow P(X=0)=\frac{1}{4}$
• $\frac{1}{2}{e}^{1t}\Rightarrow P(X=1)=\frac{1}{2}$
• $\frac{1}{4}{e}^{2t}\Rightarrow P(X=2)=\frac{1}{4}$

Probabilities sum to $\frac{1}{4}+\frac{1}{2}+\frac{1}{4}=1$ ✓, and $M_X(0)=\frac{1}{4}+\frac{1}{2}+\frac{1}{4}=1$ ✓. By uniqueness this is the distribution. (It happens to be Binomial$(2,\frac{1}{2})$: check $(1-\frac{1}{2}+\frac{1}{2}{e}^t{)}^2=(\frac{1}{2}+\frac{1}{2}{e}^t{)}^2=\frac{1}{4}+\frac{1}{2}{e}^t+\frac{1}{4}{e}^{2t}$ ✓.) Mean: $M_X^{\prime }(t)=\frac{1}{2}{e}^t+\frac{1}{2}{e}^{2t}$, so $E[X]=M_X^{\prime }(0)=1$.

The $n$-th moment is ==$M_X^{(n)}(0)$==. For independent $X,Y$: $M_{X+Y}(t)=$ ::: $M_X(t)M_Y(t)$. Poisson$(\lambda )$ MGF ::: $\exp (\lambda (e^t-1))$. Sum of independent Poissons has rate ::: ${\lambda }_1+{\lambda }_2$. $K_X(t)=\ln M_X(t)$ gives $K_X^{\prime \prime }(0)=$ ::: $\mathrm{Var}(X)$. Cauchy has no MGF because ::: $E[e^{tX}]=\infty$ for every $t\ne 0$ (heavy tails).