Exercises — Moment generating function (MGF) — definition, use

Ek jaani-pehchaani form se match karo. Parent note mein humne Bernoulli MGF $M(t)=(1-p)+p{e}^t$ derive ki thi. Yeh exactly wahi shape hai, toh $X\sim$ ==Bernoulli$(p)$==. Mean nikalo. Differentiate karo: $M_X^{\prime }(t)=p{e}^t$, toh $M_X^{\prime }(0)=p{e}^0=p$. Is tarah $E[X]=p$. Sanity check: $M_X(0)=1-p+p=1$. ✓ (Hamesha $1$ hota hai.)

Yeh Exponential$(\lambda )$ MGF hai (parent Example 1). Yeh tab hi exist karta hai jab defining integral ke andar ka exponent negative rahe, yaani $t<\lambda$; $t=\lambda$ par denominator $0$ ho jaata hai aur value blow up karti hai, aur $t>\lambda$ par integral diverge karta hai — wahan koi MGF nahi. Mean: $M_X^{\prime }(t)=\frac{\lambda }{(\lambda -t{)}^2}$, toh $E[X]=M_X^{\prime }(0)=\frac{1}{\lambda }$. $\lambda =2$ ke saath, $E[X]=\frac{1}{2}$.

Normal MGF hai $\exp (\mu t+\frac{1}{2}{\sigma }^2{t}^2)$ (parent Example 3). Term by term match karo:

• $t$ ka coefficient: $\mu =3$.
• $t^2$ ka coefficient: $\frac{1}{2}{\sigma }^2=2\Rightarrow {\sigma }^2=4$.

Toh $X\sim N(3,4)$, yaani mean $3$, variance $4$, standard deviation $2$.

Derivatives kyun? $E[X^n]=M_X^{(n)}(0)$: har derivative ek aur moment "peel off" karti hai. $M_X(t)=\lambda (\lambda -t{)}^{-1}$ likho aur differentiate karo, $\frac{d}{dt}(\lambda -t{)}^{-k}=k(\lambda -t{)}^{-k-1}$ use karke:

• $M_X^{\prime }(t)=\lambda (\lambda -t{)}^{-2}$
• $M_X^{\prime \prime }(t)=2\lambda (\lambda -t{)}^{-3}$
• $M_X^{\prime \prime \prime }(t)=6\lambda (\lambda -t{)}^{-4}$

$t=0$ set karo: $E[X^3]=M_X^{\prime \prime \prime }(0)=6\lambda \cdot {\lambda }^{-4}=\frac{6}{{\lambda }^3}$. (General pattern: $E[X^n]=\frac{n!}{{\lambda }^n}$.)

Sum set up karo. Definition se $M_X(t)={\sum }_{k=0}^{\infty }{e}^{tk}{e}^{-\lambda }\frac{{\lambda }^k}{k!}$. Regroup kyun karein? Constant $e^{-\lambda }$ bahar nikalo aur $e^{tk}{\lambda }^k=(\lambda e^t{)}^k$ merge karo: $M_X(t)=e^{-\lambda }{\sum }_{k=0}^{\infty }\frac{(\lambda e^t{)}^k}{k!}.$ Exponential series pehchano. ${\sum }_{k\ge 0}\frac{u^k}{k!}=e^u$ with $u=\lambda e^t$: $M_X(t)=e^{-\lambda }{e}^{\lambda e^t}=\exp (\lambda (e^t-1)).$ Mean. $M_X^{\prime }(t)=\lambda e^t\exp (\lambda (e^t-1))$, toh $M_X^{\prime }(0)=\lambda \cdot 1\cdot e^0=\lambda$. Is tarah $E[X]=\lambda$. Check karo $M_X(0)=\exp (\lambda (1-1))=e^0=1$. ✓

$M_X(t)=1-p+p{e}^t$, toh $M_X^{\prime }(t)=p{e}^t$ aur $M_X^{\prime \prime }(t)=p{e}^t$.

• $E[X]=M_X^{\prime }(0)=p$.
• $E[X^2]=M_X^{\prime \prime }(0)=p$.

$\mathrm{Var}(X)=E[X^2]-(E[X]{)}^2=p-p^2=p(1-p)$.

Shift & scale use karo $b=0$ ke saath: $M_Y(t)=M_X(at)=\frac{\lambda }{\lambda -at}$. Pehchaanane ke liye rewrite karo. Upar aur neeche $a$ se divide karo: $\frac{\lambda /a}{\lambda /a-t}$. $\mu =\lambda /a$ set karo: $M_Y(t)=\frac{\mu }{\mu -t},\mu =\frac{\lambda }{a}.$ Yeh rate $\mu =\lambda /a$ ke saath Exponential MGF hai, toh $Y\sim$ Exponential$(\lambda /a)$. Domain. Original ko $at<\lambda$ chahiye, yaani $t<\lambda /a=\mu$ — naye rate ke domain se match karta hai, jaisa hona chahiye.

$M_X(t)=\exp (\mu t+\frac{1}{2}{\sigma }^2{t}^2)$. Shift & scale apply karo ($a=c,b=d$): $M_Y(t)=e^{dt}{M}_X(ct)=e^{dt}\exp (\mu (ct)+\frac{1}{2}{\sigma }^2(ct{)}^2).$ Exponents combine karo (exponent mein jo hai woh add karo): $M_Y(t)=\exp ((c\mu +d)t+\frac{1}{2}(c^2{\sigma }^2)t^2).$ Yeh Normal MGF hai naye mean $c\mu +d$ aur naye variance $c^2{\sigma }^2$ ke saath. Uniqueness se, $Y\sim N(c\mu +d,c^2{\sigma }^2)$. Normal ka linear map Normal hota hai.

$0$ ke paas finiteness test karo. Kisi bhi $t>0$ ke liye, $M_X(t)={\int }_{-\infty }^{\infty }{e}^{tx}\frac{1}{\pi (1+x^2)}dx$ ki right tail dekho. Jab $x\to +\infty$, $e^{tx}$ exponentially badhta hai jabki $\frac{1}{\pi (1+x^2)}$ sirf $1/x^2$ jaisa decay karta hai. Exponential growth polynomial decay ko kuch nahi samajhta, toh integrand $\to \infty$ aur integral har $t>0$ ke liye diverge karta hai. Symmetry se $t<0$ bhi left tail par fail karta hai. Toh woh akeela $t$ jahan integral finite hai woh $t=0$ hai — lekin MGF ko $0$ ke aas-paas ek open interval par finite hona chahiye. Is liye Cauchy ka koi MGF nahi; tumhe Characteristic Function $E[e^{itX}]$ ki taraf switch karna padega, jiska $|e^{itx}|=1$ integral ko hamesha finite rakhta hai.

Independents ka sum = MGFs ka product. Har factor $\frac{\lambda }{\lambda -t}$ hai ($t<\lambda$ ke liye): $M_S(t)={\prod }_{i=1}^n\frac{\lambda }{\lambda -t}={\left(\frac{\lambda }{\lambda -t}\right)}^n,t<\lambda .$ Kyunki saare $n$ factors identical hain, product bas $n$-th power hai. Yeh Gamma$(n,\lambda )$ MGF hai — Gamma density ko kabhi integrate kiye bina nikala gaya. Shortcut se mean. $\ln M_S(t)=n\ln \lambda -n\ln (\lambda -t)$, toh $\frac{d}{dt}\ln M_S(t)=\frac{n}{\lambda -t}$, $t=0$ par $E[S]=n/\lambda$ milta hai — jaise expect tha $n$ copies se, har ek ka mean $1/\lambda$.

L2.2 se, $M_X(t)=\exp ({\lambda }_1(e^t-1))$ aur $M_Y(t)=\exp ({\lambda }_2(e^t-1))$. Multiply karo (independence): $M_{X+Y}(t)=\exp ({\lambda }_1(e^t-1))\exp ({\lambda }_2(e^t-1))=\exp (({\lambda }_1+{\lambda }_2)(e^t-1)).$ Yeh exactly Poisson MGF hai rate ${\lambda }_1+{\lambda }_2$ ke saath. Uniqueness se, $X+Y\sim$ Poisson$({\lambda }_1+{\lambda }_2)$. Poisson rates simply add ho jaati hain.

Do Normal MGFs multiply karo (independence): $M_{X+Y}(t)=\exp ({\mu }_{1}t+\frac{1}{2}{\sigma }_1^2{t}^2)\exp ({\mu }_{2}t+\frac{1}{2}{\sigma }_2^2{t}^2).$ Exponents add karo: $M_{X+Y}(t)=\exp (({\mu }_1+{\mu }_2)t+\frac{1}{2}({\sigma }_1^2+{\sigma }_2^2)t^2).$ Yeh Normal MGF hai mean ${\mu }_1+{\mu }_2$ aur variance ${\sigma }_1^2+{\sigma }_2^2$ ke saath. Toh $X+Y\sim N({\mu }_1+{\mu }_2,{\sigma }_1^2+{\sigma }_2^2)$. Means add hoti hain aur (independents ke liye) variances add hoti hain — standard deviations kabhi nahi.

Pehle scale karo. Har $X_i/\sqrt{n}$ ka MGF $M(t/\sqrt{n})$ hai (shift & scale, $a=1/\sqrt{n}$). Independence se $n$ copies par product: $M_{S_n}(t)=\left[M\right(t/\sqrt{n}){]}^n.$ $M$ ko $0$ ke paas Taylor-expand karo (parent note: $M(s)=1+E[X]s+\frac{1}{2}E[X^2]s^2+\cdots$). $E[X]=0,E[X^2]=1$ aur $s=t/\sqrt{n}$ ke saath: $M\left(\frac{t}{\sqrt{n}}\right)=1+0\cdot \frac{t}{\sqrt{n}}+\frac{1}{2}\cdot \frac{t^2}{n}+o\left(\frac{1}{n}\right)=1+\frac{t^2}{2n}+o\left(\frac{1}{n}\right).$ $n$-th power lo aur limit lo. $(1+\frac{a}{n}+o(1/n){)}^n\to e^a$ use karke $a=\frac{t^2}{2}$ ke saath: $M_{S_n}(t)\to e^{t^2/2}.$ Yeh standard Normal MGF hai. Uniqueness se, $S_n$ distribution mein $N(0,1)$ ki taraf converge karta hai — Central Limit Theorem, poori tarah $\frac{1}{2}{s}^2$ term ke $\sqrt{n}$ scaling survive karne se driven.

Log kyun lete hain? $\ln$ par chain rule $M$ ki messy derivatives ko clean central quantities mein badal deta hai. First derivative: $K_X^{\prime }(t)=\frac{M_X^{\prime }(t)}{M_X(t)}$. $t=0$ par, $M_X(0)=1$ aur $M_X^{\prime }(0)=E[X]$, toh $K_X^{\prime }(0)=\frac{E[X]}{1}=E[X].$ Second derivative (quotient rule): $K_X^{\prime \prime }(t)=\frac{M_X^{\prime \prime }(t)M_X(t)-(M_X^{\prime }(t){)}^2}{M_X(t{)}^2}.$ $t=0$ par: numerator $=E[X^2]\cdot 1-(E[X]{)}^2$, denominator $=1$, toh $K_X^{\prime \prime }(0)=E[X^2]-(E[X]{)}^2=\mathrm{Var}(X).$ Toh log-MGF ki pehli do derivatives at $0$ tumhe mean aur variance seedha de deti hain — raw moments se zyada clean.

Definition ko ulta padho. Discrete $X$ ke liye, $M_X(t)={\sum }_x{e}^{tx}P(X=x)$. Toh har term $c{e}^{tx}$ kehti hai "value $x$ ki probability $c$ hai." Match karo:

• $\frac{1}{4}{e}^{0t}\Rightarrow P(X=0)=\frac{1}{4}$
• $\frac{1}{2}{e}^{1t}\Rightarrow P(X=1)=\frac{1}{2}$
• $\frac{1}{4}{e}^{2t}\Rightarrow P(X=2)=\frac{1}{4}$

Probabilities ka sum $\frac{1}{4}+\frac{1}{2}+\frac{1}{4}=1$ ✓, aur $M_X(0)=\frac{1}{4}+\frac{1}{2}+\frac{1}{4}=1$ ✓. Uniqueness se yeh hi distribution hai. (Yeh Binomial$(2,\frac{1}{2})$ nikla: check karo $(1-\frac{1}{2}+\frac{1}{2}{e}^t{)}^2=(\frac{1}{2}+\frac{1}{2}{e}^t{)}^2=\frac{1}{4}+\frac{1}{2}{e}^t+\frac{1}{4}{e}^{2t}$ ✓.) Mean: $M_X^{\prime }(t)=\frac{1}{2}{e}^t+\frac{1}{2}{e}^{2t}$, toh $E[X]=M_X^{\prime }(0)=1$.

$n$-th moment hai ==$M_X^{(n)}(0)$==. Independent $X,Y$ ke liye: $M_{X+Y}(t)=$ ::: $M_X(t)M_Y(t)$. Poisson$(\lambda )$ MGF ::: $\exp (\lambda (e^t-1))$. Independent Poissons ka sum jiski rate ::: ${\lambda }_1+{\lambda }_2$ hai. $K_X(t)=\ln M_X(t)$ deta hai $K_X^{\prime \prime }(0)=$ ::: $\mathrm{Var}(X)$. Cauchy ka koi MGF nahi kyunki ::: $E[e^{tX}]=\infty$ har $t\ne 0$ ke liye (heavy tails).