Intuition The big idea in one breathA random variable X X X E [ X ] , E [ X 2 ] , E [ X 3 ] , … E[X], E[X^2], E[X^3], \dots E [ X ] , E [ X 2 ] , E [ X 3 ] , … MGF packs all moments into a single function M X ( t ) M_X(t) M X ( t ) t = 0 t=0 t = 0 d d t \frac{d}{dt} d t d
Definition Moment Generating FunctionFor a random variable X X X M X ( t ) = E [ e t X ] M_X(t) = E\!\left[e^{tX}\right] M X ( t ) = E [ e tX ] t t t 0 0 0 finite .
Discrete: M X ( t ) = ∑ x e t x p ( x ) M_X(t) = \sum_x e^{tx}\, p(x) M X ( t ) = ∑ x e t x p ( x )
Continuous: M X ( t ) = ∫ − ∞ ∞ e t x f ( x ) d x M_X(t) = \int_{-\infty}^{\infty} e^{tx} f(x)\, dx M X ( t ) = ∫ − ∞ ∞ e t x f ( x ) d x
If M X ( t ) M_X(t) M X ( t ) 0 0 0 X X X uniquely determined by it.
The trick is the Taylor series of e t X e^{tX} e tX . Derive it from scratch:
e t X = ∑ n = 0 ∞ ( t X ) n n ! = 1 + t X + t 2 X 2 2 ! + t 3 X 3 3 ! + ⋯ e^{tX} = \sum_{n=0}^{\infty} \frac{(tX)^n}{n!} = 1 + tX + \frac{t^2X^2}{2!} + \frac{t^3X^3}{3!} + \cdots e tX = ∑ n = 0 ∞ n ! ( tX ) n = 1 + tX + 2 ! t 2 X 2 + 3 ! t 3 X 3 + ⋯
Take expectation of both sides (linearity of E E E
M X ( t ) = E [ e t X ] = 1 + t E [ X ] + t 2 2 ! E [ X 2 ] + t 3 3 ! E [ X 3 ] + ⋯ M_X(t) = E[e^{tX}] = 1 + tE[X] + \frac{t^2}{2!}E[X^2] + \frac{t^3}{3!}E[X^3] + \cdots M X ( t ) = E [ e tX ] = 1 + tE [ X ] + 2 ! t 2 E [ X 2 ] + 3 ! t 3 E [ X 3 ] + ⋯
must work
The coefficient of t n t^n t n E [ X n ] n ! \dfrac{E[X^n]}{n!} n ! E [ X n ] t n t^n t n M X ( n ) ( 0 ) n ! \dfrac{M_X^{(n)}(0)}{n!} n ! M X ( n ) ( 0 ) E [ X n ] = M X ( n ) ( 0 ) E[X^n] = M_X^{(n)}(0) E [ X n ] = M X ( n ) ( 0 ) designed so that each power of t t t
So:
M X ( 0 ) = 1 M_X(0) = 1 M X ( 0 ) = 1 E [ e 0 ] = E [ 1 ] E[e^0]=E[1] E [ e 0 ] = E [ 1 ] Always check this first. M X ′ ( 0 ) = E [ X ] M_X'(0) = E[X] M X ′ ( 0 ) = E [ X ] M X ′ ′ ( 0 ) = E [ X 2 ] M_X''(0) = E[X^2] M X ′′ ( 0 ) = E [ X 2 ] Var ( X ) = M X ′ ′ ( 0 ) − ( M X ′ ( 0 ) ) 2 \operatorname{Var}(X) = M_X''(0) - \big(M_X'(0)\big)^2 Var ( X ) = M X ′′ ( 0 ) − ( M X ′ ( 0 ) ) 2
Worked example Example 1 — Exponential( λ ) (\lambda) ( λ ) f ( x ) = λ e − λ x f(x) = \lambda e^{-\lambda x} f ( x ) = λ e − λ x x ≥ 0 x\ge 0 x ≥ 0 M X ( t ) = ∫ 0 ∞ e t x λ e − λ x d x = λ ∫ 0 ∞ e − ( λ − t ) x d x = λ λ − t , t < λ . M_X(t) = \int_0^\infty e^{tx}\lambda e^{-\lambda x}\,dx = \lambda\int_0^\infty e^{-(\lambda - t)x}\,dx = \frac{\lambda}{\lambda - t},\quad t<\lambda. M X ( t ) = ∫ 0 ∞ e t x λ e − λ x d x = λ ∫ 0 ∞ e − ( λ − t ) x d x = λ − t λ , t < λ .
Why t < λ t<\lambda t < λ The integral converges only if the exponent − ( λ − t ) -(\lambda-t) − ( λ − t ) λ − t > 0 \lambda - t>0 λ − t > 0 M X ′ ( t ) = λ ( λ − t ) 2 ⇒ M X ′ ( 0 ) = 1 λ = E [ X ] . M_X'(t) = \dfrac{\lambda}{(\lambda - t)^2}\Rightarrow M_X'(0)=\dfrac{1}{\lambda}=E[X]. M X ′ ( t ) = ( λ − t ) 2 λ ⇒ M X ′ ( 0 ) = λ 1 = E [ X ] . M X ′ ′ ( t ) = 2 λ ( λ − t ) 3 ⇒ M X ′ ′ ( 0 ) = 2 λ 2 = E [ X 2 ] . M_X''(t)=\dfrac{2\lambda}{(\lambda-t)^3}\Rightarrow M_X''(0)=\dfrac{2}{\lambda^2}=E[X^2]. M X ′′ ( t ) = ( λ − t ) 3 2 λ ⇒ M X ′′ ( 0 ) = λ 2 2 = E [ X 2 ] . Var = 2 λ 2 − 1 λ 2 = 1 λ 2 . \operatorname{Var}=\dfrac{2}{\lambda^2}-\dfrac{1}{\lambda^2}=\dfrac{1}{\lambda^2}. Var = λ 2 2 − λ 2 1 = λ 2 1 .
Worked example Example 2 — Bernoulli( p ) (p) ( p ) ( n , p ) (n,p) ( n , p ) X ∈ { 0 , 1 } X\in\{0,1\} X ∈ { 0 , 1 } P ( 1 ) = p P(1)=p P ( 1 ) = p M X ( t ) = e t ⋅ 0 ( 1 − p ) + e t ⋅ 1 p = ( 1 − p ) + p e t = 1 − p + p e t . M_X(t)=e^{t\cdot 0}(1-p)+e^{t\cdot 1}p = (1-p)+pe^t = 1-p+pe^t. M X ( t ) = e t ⋅ 0 ( 1 − p ) + e t ⋅ 1 p = ( 1 − p ) + p e t = 1 − p + p e t . Why this step: just sum e t x p ( x ) e^{tx}p(x) e t x p ( x ) M X ′ ( t ) = p e t ⇒ M X ′ ( 0 ) = p = E [ X ] M_X'(t)=pe^t\Rightarrow M_X'(0)=p=E[X] M X ′ ( t ) = p e t ⇒ M X ′ ( 0 ) = p = E [ X ]
Binomial= = = n n n Property 2 :
M Bin ( t ) = ( 1 − p + p e t ) n . M_{\text{Bin}}(t)=\big(1-p+pe^t\big)^n. M Bin ( t ) = ( 1 − p + p e t ) n . Why this is the payoff: we never touched the binomial pmf; multiplying MGFs of the parts gave us the whole instantly.
Worked example Example 3 — Normal( μ , σ 2 ) (\mu,\sigma^2) ( μ , σ 2 ) Z ∼ N ( 0 , 1 ) Z\sim N(0,1) Z ∼ N ( 0 , 1 ) M Z ( t ) = ∫ − ∞ ∞ e t z 1 2 π e − z 2 / 2 d z . M_Z(t)=\int_{-\infty}^\infty e^{tz}\frac{1}{\sqrt{2\pi}}e^{-z^2/2}\,dz. M Z ( t ) = ∫ − ∞ ∞ e t z 2 π 1 e − z 2 /2 d z . Why complete the square? Combine exponents: t z − z 2 2 = − 1 2 ( z 2 − 2 t z ) = − 1 2 ( ( z − t ) 2 − t 2 ) tz-\tfrac{z^2}{2} = -\tfrac12(z^2-2tz)= -\tfrac12\big((z-t)^2 - t^2\big) t z − 2 z 2 = − 2 1 ( z 2 − 2 t z ) = − 2 1 ( ( z − t ) 2 − t 2 ) M Z ( t ) = e t 2 / 2 ∫ − ∞ ∞ 1 2 π e − ( z − t ) 2 / 2 d z ⏟ = 1 ( it’s a N ( t , 1 ) density ) = e t 2 / 2 . M_Z(t)=e^{t^2/2}\underbrace{\int_{-\infty}^\infty \frac{1}{\sqrt{2\pi}}e^{-(z-t)^2/2}\,dz}_{=\,1\ (\text{it's a }N(t,1)\text{ density})}=e^{t^2/2}. M Z ( t ) = e t 2 /2 = 1 ( it’s a N ( t , 1 ) density ) ∫ − ∞ ∞ 2 π 1 e − ( z − t ) 2 /2 d z = e t 2 /2 . X = μ + σ Z X=\mu+\sigma Z X = μ + σ Z Property 1 (a = σ , b = μ a=\sigma,b=\mu a = σ , b = μ M X ( t ) = e μ t M Z ( σ t ) = exp ( μ t + 1 2 σ 2 t 2 ) . M_X(t)=e^{\mu t}M_Z(\sigma t)=\exp\!\Big(\mu t + \tfrac12\sigma^2 t^2\Big). M X ( t ) = e μ t M Z ( σ t ) = exp ( μ t + 2 1 σ 2 t 2 ) . M X ′ ( 0 ) = μ M_X'(0)=\mu M X ′ ( 0 ) = μ M X ′ ′ ( 0 ) = μ 2 + σ 2 ⇒ Var = σ 2 M_X''(0)=\mu^2+\sigma^2\Rightarrow \operatorname{Var}=\sigma^2 M X ′′ ( 0 ) = μ 2 + σ 2 ⇒ Var = σ 2
M X ( t ) = E [ e t X ] M_X(t)=E[e^{tX}] M X ( t ) = E [ e tX ] t = 1 t=1 t = 1 Why it feels right: e 1 ⋅ X = e X e^{1\cdot X}=e^X e 1 ⋅ X = e X X X X The fix: moments come from derivatives at t = 0 t=0 t = 0 , not from plugging in t = 1 t=1 t = 1 t t t bookkeeping dummy , not a value of X X X t = 0 t=0 t = 0
Common mistake "Every random variable has an MGF."Why it feels right: every variable has moments... sometimes.
The fix: M X ( t ) M_X(t) M X ( t ) finite in an interval around 0 . Heavy-tailed variables (e.g. Cauchy, or Lognormal for t > 0 t>0 t > 0 E [ e t X ] = ∞ E[e^{tX}]=\infty E [ e tX ] = ∞ characteristic function E [ e i t X ] E[e^{itX}] E [ e i tX ]
M X + Y = M X ⋅ M Y M_{X+Y}=M_X\cdot M_Y M X + Y = M X ⋅ M Y X , Y X,Y X , Y Why it feels right: the algebra looks automatic.
The fix: it needs independence . E [ e t X e t Y ] = E [ e t X ] E [ e t Y ] E[e^{tX}e^{tY}]=E[e^{tX}]E[e^{tY}] E [ e tX e t Y ] = E [ e tX ] E [ e t Y ] X ⊥ Y X\perp Y X ⊥ Y
Common mistake "Forgetting to verifyM X ( 0 ) = 1 M_X(0)=1 M X ( 0 ) = 1 Why it bites: an algebra slip often makes M X ( 0 ) ≠ 1 M_X(0)\ne1 M X ( 0 ) = 1 M X ( 0 ) = E [ 1 ] = 1 M_X(0)=E[1]=1 M X ( 0 ) = E [ 1 ] = 1
The 20% that gives 80% of the value:
Compute mean/variance without grinding integrals — just differentiate.Identify a distribution by recognizing its MGF (uniqueness).Find the distribution of a sum of independents (multiply MGFs).Prove limit theorems (CLT proof: show MGFs converge to e t 2 / 2 e^{t^2/2} e t 2 /2
Recall Feynman: explain to a 12-year-oldImagine X X X moments . Writing them one by one is slow. The MGF is a magic box : you feed it a dial number t t t n n n n n n n n n two boxes multiplied — that's why mathematicians love it.
"M ake G moments F all out by D ifferentiating at 0 ."
And the shape rule: D erivative-at-0 = moment; M ultiply MGFs = add (independent) variables.
Pun: MGF = "Moment Genie's Formula" — rub it (d d t \frac{d}{dt} d t d n n n n n n
Definition of the MGF of X X X M X ( t ) = E [ e t X ] M_X(t)=E[e^{tX}] M X ( t ) = E [ e tX ] , finite in an interval around
t = 0 t=0 t = 0 .
How do you get the n n n E [ X n ] = M X ( n ) ( 0 ) E[X^n]=M_X^{(n)}(0) E [ X n ] = M X ( n ) ( 0 ) , the
n n n -th derivative evaluated at
t = 0 t=0 t = 0 .
Why does e t X e^{tX} e tX Its Taylor series is
∑ t n X n n ! \sum \frac{t^n X^n}{n!} ∑ n ! t n X n ; taking
E E E makes the coefficient of
t n t^n t n equal
E [ X n ] / n ! E[X^n]/n! E [ X n ] / n ! .
What is M X ( 0 ) M_X(0) M X ( 0 ) 1 1 1 , since
E [ e 0 ] = E [ 1 ] = 1 E[e^{0}]=E[1]=1 E [ e 0 ] = E [ 1 ] = 1 — use it as a sanity check.
MGF of Y = a X + b Y=aX+b Y = a X + b M Y ( t ) = e b t M X ( a t ) M_Y(t)=e^{bt}M_X(at) M Y ( t ) = e b t M X ( a t ) .
MGF of X + Y X+Y X + Y X ⊥ Y X\perp Y X ⊥ Y M X + Y ( t ) = M X ( t ) M Y ( t ) M_{X+Y}(t)=M_X(t)M_Y(t) M X + Y ( t ) = M X ( t ) M Y ( t ) (multiplication, requires independence).
Variance in terms of the MGF? Var ( X ) = M X ′ ′ ( 0 ) − ( M X ′ ( 0 ) ) 2 \operatorname{Var}(X)=M_X''(0)-\big(M_X'(0)\big)^2 Var ( X ) = M X ′′ ( 0 ) − ( M X ′ ( 0 ) ) 2 .
MGF of Exponential( λ ) (\lambda) ( λ ) M X ( t ) = λ λ − t M_X(t)=\frac{\lambda}{\lambda-t} M X ( t ) = λ − t λ for
t < λ t<\lambda t < λ .
MGF of Normal( μ , σ 2 ) (\mu,\sigma^2) ( μ , σ 2 ) M X ( t ) = exp ( μ t + 1 2 σ 2 t 2 ) M_X(t)=\exp(\mu t+\tfrac12\sigma^2 t^2) M X ( t ) = exp ( μ t + 2 1 σ 2 t 2 ) .
MGF of Binomial( n , p ) (n,p) ( n , p ) ( 1 − p + p e t ) n (1-p+pe^t)^n ( 1 − p + p e t ) n , because it is a sum of
n n n independent Bernoulli MGFs multiplied together.
Why might an MGF not exist? If
E [ e t X ] = ∞ E[e^{tX}]=\infty E [ e tX ] = ∞ near
0 0 0 (heavy tails, e.g. Cauchy); then use the characteristic function instead.
What does uniqueness of MGF give you? If two MGFs agree on an interval around 0, the two distributions are identical.
Probability Distributions — every named distribution has a signature MGF.Expectation and Variance — MGF derivatives recover these directly.Characteristic Function — the E [ e i t X ] E[e^{itX}] E [ e i tX ] Central Limit Theorem — proved via convergence of MGFs to e t 2 / 2 e^{t^2/2} e t 2 /2 Independence (Probability) — the condition behind the multiplication rule.Taylor Series — the engine that makes moments "fall out."Cumulant Generating Function — ln M X ( t ) \ln M_X(t) ln M X ( t )
property, needs independence
convolution to multiplication
M_X t = sum t^n E X^n over n!
Shift and scale e^bt M_X at
Sum of independents product of MGFs
Uniqueness identifies distribution
Intuition Hinglish mein samjhoDekho, kisi bhi random variable X X X E [ X ] E[X] E [ X ] E [ X 2 ] E[X^2] E [ X 2 ] E [ X 3 ] E[X^3] E [ X 3 ] M X ( t ) = E [ e t X ] M_X(t)=E[e^{tX}] M X ( t ) = E [ e tX ] e t X e^{tX} e tX 1 + t X + t 2 X 2 2 ! + … 1+tX+\frac{t^2X^2}{2!}+\dots 1 + tX + 2 ! t 2 X 2 + … t n t^n t n E [ X n ] / n ! E[X^n]/n! E [ X n ] / n !
Moment nikalna ho to bas t = 0 t=0 t = 0 E [ X n ] = M X ( n ) ( 0 ) E[X^n]=M_X^{(n)}(0) E [ X n ] = M X ( n ) ( 0 ) E [ X 2 ] E[X^2] E [ X 2 ] M X ( 0 ) = 1 M_X(0)=1 M X ( 0 ) = 1 t = 1 t=1 t = 1
Do superpowers hain MGF ke. Pehla: Y = a X + b Y=aX+b Y = a X + b M Y ( t ) = e b t M X ( a t ) M_Y(t)=e^{bt}M_X(at) M Y ( t ) = e b t M X ( a t ) X X X Y Y Y M X + Y ( t ) = M X ( t ) M Y ( t ) M_{X+Y}(t)=M_X(t)M_Y(t) M X + Y ( t ) = M X ( t ) M Y ( t ) n n n
Lekin yaad rakho — har variable ka MGF exist nahi karta (jaise Cauchy ka, kyunki integral infinite ho jaata hai). Aise case me characteristic function E [ e i t X ] E[e^{itX}] E [ e i tX ]
