Moment generating function (MGF) — definition, use: Visual walkthrough

We assume only that you know: a random variable $X$ is a number that comes out random, and $E [\cdot]$ ("the expected value") is the long-run average. Everything else — even what a Taylor series is — we grow below.

Step 1 — What is a "moment", pictured?

WHAT. We line up the first few moments as a strip of numbers describing the shape of $X$ .

WHY. The whole point of the MGF is to pack this infinite strip into one function. So first we must see the strip we are trying to pack.

PICTURE. Figure s01: a clay blob (a distribution) with its balance point (moment 1), its width (moment 2), and its lean (moment 3) called out. The moments are an infinite to-do list of facts about the blob.

Figure — Moment generating function (MGF) — definition, use

Step 2 — What is $e^{tX}$ , and why an exponential?

Before packing anything we need the packing material. That material is $e^{tX}$ . Let us earn every symbol.

Now let $u = tX$ , where $t$ is a plain real number we get to dial (a knob), and $X$ is our random variable:

$e^{tX} = 1 + power t^{1} tX + power t^{2} \frac{t ^{2} X ^{2}}{2 !} + power t^{3} \frac{t ^{3} X ^{3}}{3 !} + \dots$

WHAT. We stretched the exponential into an infinite polynomial, then substituted $u = tX$ .

WHY this tool and not another? Look at the right-hand side: the term with $t^{n}$ carries exactly one power $X^{n}$ — never mixed. No other elementary function separates the powers of $X$ so cleanly into separate "shelves" labelled by $t^{0}, t^{1}, t^{2}, \dots$ . That clean shelving is the whole trick. (Contrast: $sin (tX)$ or $\frac{1}{1 - tX}$ mix powers in messier ways or blow up.)

PICTURE. Figure s02: the exponential curve drawn as a stack of transparent polynomial pieces $1$ , then $+ u$ , then $+ u^{2} /2$ , ... each a coloured layer, showing how they add up to the smooth curve.

Step 3 — Averaging the series: moments climb onto the shelves

Take the expectation $E [\cdot]$ of both sides. Because $E$ is linear — the average of a sum is the sum of the averages, and constants like $t^{n} / n!$ slide out — we may average term by term:

$M_{X} (t) = E [e^{tX}] = E [1] 1 + t moment 1 E [X] + \frac{t ^{2}}{2 !} moment 2 E [X^{2}] + \frac{t ^{3}}{3 !} moment 3 E [X^{3}] + \dots$

WHAT. We averaged, and the random $X^{n}$ on each shelf turned into the fixed number $E [X^{n}]$ .

WHY. This is the moment of victory: the entire infinite strip of moments from Step 1 is now packed into a single function $M_{X} (t)$ . Each moment rides on its own power of $t$ .

PICTURE. Figure s03: the moment strip from s01 being loaded, one number per shelf, onto a shelf-rack labelled $t^{0}, t^{1}, t^{2}, \dots$ . This uses Taylor-series shelving and linearity of expectation.

Step 4 — Reading one shelf: differentiate

We packed everything onto shelves; now we need a tool that reads one shelf and only one. That tool is the derivative $\frac{d}{d t}$ (the slope / rate-of-change operator) together with evaluation at $t = 0$ .

Watch what differentiation does to a single shelf $\frac{E [ X ^{n} ]}{n !} t^{n}$ :

$\frac{d}{d t} [\frac{E [ X ^{n} ]}{n !} t^{n}] = \frac{E [ X ^{n} ]}{n !} \cdot n t^{n - 1} = \frac{E [ X ^{n} ]}{( n - 1 )!} t^{n - 1} .$

Each differentiation pulls down the power by one ( $t^{n} \to t^{n - 1}$ ) and multiplies by that power.

WHAT. We used $\frac{d}{d t}$ as a "shelf-lowering" crank and $t = 0$ as an "erase-the-rest" switch.

WHY. No single algebraic step can pick out one coefficient of an infinite polynomial — but repeated differentiation followed by $t = 0$ can. That is precisely the question the derivative answers here: "what is the coefficient on this power?"

PICTURE. Figure s04: a comb of shelves; the $n = 2$ shelf is dragged down two notches to the floor (becoming $E [X^{2}]$ ) while a " $t = 0$ " guillotine erases every shelf that still floats above zero.

Step 5 — The master formula assembled

Do the crank $n$ times on the whole series and set $t = 0$ ; only shelf $n$ survives:

$\frac{d ^{n}}{d t ^{n}} M_{X} (t)_{t = 0} = E [X^{n}] .$

The first two cranks are the ones you use daily:

$M_{X}^{'} (0) = E [X] (the mean), M_{X}^{''} (0) = E [X^{2}] .$

And from those, variance (the spread from Expectation and Variance):

$Var (X) = E [X^{2}] M_{X}^{''} (0) - (E [X])^{2} (M_{X}^{'} (0))^{2} .$

WHAT. We collapsed Steps 2–4 into a one-line recipe.

WHY. This is the machine the parent promised — turn the crank $n$ times, out pops moment $n$ .

PICTURE. Figure s05: the "moment genie" box. Dial set to $t$ , crank turned $n$ times, dial snapped back to $0$ , and moment $n$ drops out of the chute.

Step 6 — A worked crank: Exponential $(λ)$

Let us watch the machine run on a concrete $X \sim Exponential (λ)$ , density $f (x) = λ e^{- λ x}$ for $x \geq 0$ .

$M_{X} (t) = \int_{0}^{\infty} e^{t x} λ e^{- λ x} d x = λ \int_{0}^{\infty} e^{- (λ - t) x} d x = \frac{λ}{λ - t}, t < λ .$

The exponent $- (λ - t)$ must be negative for the area to be finite, forcing $t < λ$ . That is the "interval around $0$ " being real.

Crank once and twice: $M_{X}^{'} (t) = \frac{λ}{( λ - t ) ^{2}} \Rightarrow M_{X}^{'} (0) = \frac{1}{λ} = E [X], M_{X}^{''} (t) = \frac{2 λ}{( λ - t ) ^{3}} \Rightarrow M_{X}^{''} (0) = \frac{2}{λ ^{2}} = E [X^{2}] .$ $Var (X) = \frac{2}{λ ^{2}} - \frac{1}{λ ^{2}} = \frac{1}{λ ^{2}} . ✓$

WHAT / WHY. We fed a real distribution in; the crank gave mean and variance with no integration of $x^{2}$ — the packing paid off.

PICTURE. Figure s06: the curve $M_{X} (t) = λ / (λ - t)$ with a vertical wall (asymptote) at $t = λ$ , the point $M_{X} (0) = 1$ marked, and the slope at $t = 0$ drawn as the tangent whose value is $1/ λ$ .

Step 7 — Edge & degenerate cases (never get surprised)

WHAT. We checked the dial-at-zero identity, the collapsed-blob limit, and the case where the box can't be built.

WHY. A machine you trust must behave at its boundaries; case 1 catches errors, case 2 shows the smallest input, case 3 shows the failure mode and its rescue.

PICTURE. Figure s07: three mini-panels — (a) every MGF passing through the same anchor point $(0, 1)$ ; (b) a spike distribution at $x = c$ giving $e^{t c}$ ; (c) a heavy tail whose $e^{t x}$ area explodes, crossed out.

Step 8 — Bonus crank: multiplication = gluing blobs

The same packed function has a second superpower. For independent $X$ and $Y$ (see Independence (Probability)):

$M_{X + Y} (t) = E [e^{t (X + Y)}] = E [e^{tX} e^{t Y}] = ⊥ E [e^{tX}] E [e^{t Y}] = M_{X} (t) M_{Y} (t) .$

The single splitting step marked $⊥$ needs independence — that is what lets the average of a product become the product of averages.

WHY it matters. Adding independent variables would normally require a messy convolution of densities; here it is plain multiplication. Chaining this ( $M \to M^{n}$ ) and taking limits is exactly how the Central Limit Theorem is proven, and taking $lo g$ of $M$ links to the Cumulant Generating Function.

PICTURE. Figure s08: two blobs glued into one; their packed boxes multiply into the box of the sum.

The one-picture summary

Figure s09 compresses the whole story: moments (left) get packed by the exponential shelves into one curve $M_{X} (t)$ (center); a crank + reset-to-zero (right) reads any single moment back out; and two boxes multiply when blobs are glued.

Recall Feynman retelling of the whole walkthrough

A blob of clay has an endless list of facts: where its middle sits, how wide it is, how it leans, and on forever. That is our moment strip (Step 1). To carry the list without dropping any, we build a magic box out of the exponential $e^{tX}$ , because when you unfold that exponential into an infinite polynomial each power of the dial $t$ sits on its own private shelf (Step 2), and each shelf holds exactly one moment once we average (Step 3). To read a fact back, we spin the derivative crank: each spin lowers a shelf by one notch, and when we finally snap the dial back to zero, every shelf but the one we wanted vanishes — that leaves a single clean moment (Steps 4–5). We tested the box on a real exponential distribution and out came the mean $1/ λ$ and variance $1/ λ^{2}$ with no ugly integrals (Step 6). We checked the corners: the dial at zero always reads $1$ , a frozen constant blob has zero spread, and a wild heavy-tailed blob overflows the box so we switch to its cousin the characteristic function (Step 7). Finally, glue two independent blobs together and their boxes simply multiply — that one fact powers half of probability theory (Step 8).

Recall Quick self-check

Why must we set $t = 0$ after differentiating, not $t = 1$ ? ::: Setting $t = 0$ kills every shelf except the one lowered to a constant; $t = 1$ would leave all the other shelves alive and mix moments together. What guarantees $M_{X} (0) = 1$ for every $X$ ? ::: $M_{X} (0) = E [e^{0}] = E [1] = 1$ . Which single assumption lets $M_{X + Y} = M_{X} M_{Y}$ ? ::: Independence of $X$ and $Y$ .