Moment generating function (MGF) — definition, use: Visual walkthrough

4.9.5 · D2 · Maths › Probability Theory & Statistics › Moment generating function (MGF) — definition, use

Hum sirf yeh maante hain ki aap jaante ho: ek random variable $X$ ek aisa number hai jo randomly aata hai, aur $E [\cdot]$ ("expected value") long-run average hai. Baaki sab — yahan tak ki Taylor series kya hoti hai — hum neeche khud develop karenge.

Step 1 — "Moment" kya hota hai, picture mein?

KYA. Hum pehle kuch moments ko ek number strip ki tarah likhte hain jo $X$ ki shape describe karti hai.

KYU. MGF ka poora point yahi hai ki is infinite strip ko ek function mein pack karo. Toh pehle woh strip dekhna zaroori hai jo hum pack karne wale hain.

PICTURE. Figure s01: ek clay blob (ek distribution) jisme uska balance point (moment 1), uski width (moment 2), aur uski lean (moment 3) dikhaye gaye hain. Moments us blob ke baare mein facts ki ek infinite to-do list hain.

Figure — Moment generating function (MGF) — definition, use

Step 2 — $e^{tX}$ kya hai, aur exponential hi kyun?

Kuch pack karne se pehle hume packing material chahiye. Woh material hai $e^{tX}$ . Aao har symbol ko samjhein.

Ab $u = tX$ lo, jahan $t$ ek plain real number hai jise hum dial kar sakte hain (ek knob), aur $X$ humara random variable hai:

$e^{tX} = 1 + power t^{1} tX + power t^{2} \frac{t ^{2} X ^{2}}{2 !} + power t^{3} \frac{t ^{3} X ^{3}}{3 !} + \dots$

KYA. Humne exponential ko infinite polynomial mein stretch kiya, phir $u = tX$ substitute kiya.

YEH TOOL HI KYU, koi aur kyun nahi? Right-hand side dekho: $t^{n}$ wali term mein exactly ek power $X^{n}$ hai — kabhi mixed nahi. Koi aur elementary function, $X$ ki powers ko alag-alag "shelves" pe itni cleanly nahi rakhta jaise $t^{0}, t^{1}, t^{2}, \dots$ pe. Wahi clean shelving poori trick hai. (Compare karo: $sin (tX)$ ya $\frac{1}{1 - tX}$ powers ko messier tarike se mix karte hain ya blow up ho jaate hain.)

PICTURE. Figure s02: exponential curve ko transparent polynomial pieces $1$ , phir $+ u$ , phir $+ u^{2} /2$ , ... ki stack ki tarah draw kiya gaya hai — har ek coloured layer — dikhate hue ki woh smooth curve mein kaise add up hote hain.

Step 3 — Series ka average lena: moments shelves pe chadh jaate hain

Dono sides ka expectation $E [\cdot]$ lo. Kyunki $E$ linear hai — sum ka average, averages ka sum hota hai, aur constants jaise $t^{n} / n!$ bahar aa jaate hain — hum term by term average le sakte hain:

$M_{X} (t) = E [e^{tX}] = E [1] 1 + t moment 1 E [X] + \frac{t ^{2}}{2 !} moment 2 E [X^{2}] + \frac{t ^{3}}{3 !} moment 3 E [X^{3}] + \dots$

KYA. Humne average liya, aur har shelf pe random $X^{n}$ fixed number $E [X^{n}]$ ban gaya.

KYU. Yahi jeet ka moment hai: Step 1 se moments ki poori infinite strip ab ek single function $M_{X} (t)$ mein pack ho gayi. Har moment apni khud ki $t$ ki power pe sawaar hai.

PICTURE. Figure s03: s01 wali moment strip ek ek number ko $t^{0}, t^{1}, t^{2}, \dots$ labelled shelf-rack pe load karti hui dikhaayi gayi hai. Isme Taylor-series shelving aur linearity of expectation use hoti hai.

Step 4 — Ek shelf padhna: differentiate karo

Humne sab kuch shelves pe pack kar diya; ab hume ek aisa tool chahiye jo sirf ek shelf padhta ho. Woh tool hai derivative $\frac{d}{d t}$ (slope / rate-of-change operator) aur saath mein $t = 0$ pe evaluation.

Dekho ki differentiation ek single shelf $\frac{E [ X ^{n} ]}{n !} t^{n}$ ke saath kya karta hai:

$\frac{d}{d t} [\frac{E [ X ^{n} ]}{n !} t^{n}] = \frac{E [ X ^{n} ]}{n !} \cdot n t^{n - 1} = \frac{E [ X ^{n} ]}{( n - 1 )!} t^{n - 1} .$

Har differentiation power ko ek se kam karta hai ( $t^{n} \to t^{n - 1}$ ) aur us power se multiply karta hai.

KYA. Humne $\frac{d}{d t}$ ko "shelf-lowering" crank ki tarah use kiya aur $t = 0$ ko "baaki erase karo" switch ki tarah.

KYU. Koi single algebraic step ek infinite polynomial ka ek coefficient pick out nahi kar sakta — lekin repeated differentiation ke baad $t = 0$ kar sakta hai. Yahi woh sawaal hai jo derivative yahan answer karta hai: "is power ka coefficient kya hai?"

PICTURE. Figure s04: shelves ki ek comb; $n = 2$ wali shelf do notches neeche floor tak khinchi jaati hai ( $E [X^{2}]$ ban jaati hai) jabki ek " $t = 0$ " guillotine har shelf ko erase kar deti hai jo abhi bhi zero ke upar float kar rahi hai.

Step 5 — Master formula assembled

Poori series pe $n$ baar crank karo aur $t = 0$ set karo; sirf shelf $n$ bachti hai:

$\frac{d ^{n}}{d t ^{n}} M_{X} (t)_{t = 0} = E [X^{n}] .$

Pehle do cranks woh hain jo aap daily use karte ho:

$M_{X}^{'} (0) = E [X] (the mean), M_{X}^{''} (0) = E [X^{2}] .$

Aur unse, variance (Expectation and Variance se spread):

$Var (X) = E [X^{2}] M_{X}^{''} (0) - (E [X])^{2} (M_{X}^{'} (0))^{2} .$

KYA. Humne Steps 2–4 ko ek one-line recipe mein compress kar diya.

KYU. Yahi woh machine hai jo parent note ne promise ki thi — crank $n$ baar ghuma, moment $n$ bahar aata hai.

PICTURE. Figure s05: "moment genie" box. Dial $t$ pe set hai, crank $n$ baar ghuma, dial $0$ pe snap back, aur chute se moment $n$ bahar aata hai.

Step 6 — Ek worked crank: Exponential $(λ)$

Aao machine ko ek concrete $X \sim Exponential (λ)$ pe chalate hue dekhein, density $f (x) = λ e^{- λ x}$ for $x \geq 0$ .

$M_{X} (t) = \int_{0}^{\infty} e^{t x} λ e^{- λ x} d x = λ \int_{0}^{\infty} e^{- (λ - t) x} d x = \frac{λ}{λ - t}, t < λ .$

Exponent $- (λ - t)$ negative hona chahiye taaki area finite ho, isliye $t < λ$ zaroori hai. Yahi woh "interval around $0$ " hai jo actually real hai.

Ek baar aur do baar crank karo: $M_{X}^{'} (t) = \frac{λ}{( λ - t ) ^{2}} \Rightarrow M_{X}^{'} (0) = \frac{1}{λ} = E [X], M_{X}^{''} (t) = \frac{2 λ}{( λ - t ) ^{3}} \Rightarrow M_{X}^{''} (0) = \frac{2}{λ ^{2}} = E [X^{2}] .$ $Var (X) = \frac{2}{λ ^{2}} - \frac{1}{λ ^{2}} = \frac{1}{λ ^{2}} . ✓$

KYA / KYU. Humne ek real distribution dali; crank ne mean $1/ λ$ aur variance bina kisi ugly integral ke de diya — packing ka fayda mila.

PICTURE. Figure s06: curve $M_{X} (t) = λ / (λ - t)$ ek vertical wall (asymptote) ke saath $t = λ$ pe, point $M_{X} (0) = 1$ marked hai, aur $t = 0$ pe tangent ke slope ki value $1/ λ$ dikhaayi gayi hai.

Step 7 — Edge & degenerate cases (kabhi surprise mat lo)

KYA. Humne dial-at-zero identity, collapsed-blob limit, aur woh case check kiya jahan box ban hi nahi sakta.

KYU. Jis machine pe aap trust karte ho woh apni boundaries pe sahi behave karni chahiye; case 1 errors pakadta hai, case 2 sabse chhota input dikhata hai, case 3 failure mode aur uska rescue dikhata hai.

PICTURE. Figure s07: teen mini-panels — (a) har MGF ek hi anchor point $(0, 1)$ se guzarti hai; (b) $x = c$ pe ek spike distribution $e^{t c}$ deti hai; (c) ek heavy tail jiska $e^{t x}$ area explode ho jaata hai, crossed out.

Step 8 — Bonus crank: multiplication = blobs ko glue karna

Isi packed function mein ek doosri superpower bhi hai. Independent $X$ aur $Y$ ke liye (dekho Independence (Probability)):

$M_{X + Y} (t) = E [e^{t (X + Y)}] = E [e^{tX} e^{t Y}] = ⊥ E [e^{tX}] E [e^{t Y}] = M_{X} (t) M_{Y} (t) .$

$⊥$ se marked yeh single splitting step independence maangta hai — yahi woh cheez hai jo average of a product ko product of averages banna deta hai.

KYU yeh important hai. Independent variables ko add karne ke liye normally densities ki messy convolution chahiye hoti; yahan yeh simple multiplication hai. Ise chain karna ( $M \to M^{n}$ ) aur limits lena exactly woh hai jisse Central Limit Theorem prove hoti hai, aur $M$ ka $lo g$ lena Cumulant Generating Function se link karta hai.

PICTURE. Figure s08: do blobs ek mein glue ho rahe hain; unke packed boxes sum ke box mein multiply ho jaate hain.

Ek-picture summary

Figure s09 poori kahani compress karta hai: moments (left) exponential shelves se ek curve $M_{X} (t)$ (center) mein pack hote hain; ek crank + reset-to-zero (right) koi bhi single moment wapas padh leta hai; aur jab blobs glue hote hain toh do boxes multiply karte hain.

Recall Poore walkthrough ki Feynman retelling

Clay ke ek blob mein facts ki ek endless list hoti hai: uska middle kahan hai, kitna wide hai, kaise lean karta hai, aur yeh forever chalta rehta hai. Yahi humari moment strip hai (Step 1). List ko bina giraaye carry karne ke liye, hum exponential $e^{tX}$ se ek magic box banate hain, kyunki jab aap us exponential ko ek infinite polynomial mein unfold karte ho toh dial $t$ ki har power apni private shelf pe hoti hai (Step 2), aur average lete hi har shelf exactly ek moment hold karti hai (Step 3). Koi fact wapas padhne ke liye, hum derivative crank ghmaate hain: har spin ek shelf ko ek notch neeche karta hai, aur jab hum finally dial zero pe snap karte hain, toh jis ek shelf ko chahiye tha uske siwa sab gayab ho jaata hai — ek clean moment bachta hai (Steps 4–5). Humne box ko ek real exponential distribution pe test kiya aur bina kisi ugly integral ke mean $1/ λ$ aur variance $1/ λ^{2}$ nikal aaya (Step 6). Humne corners check kiye: dial zero pe hamesha $1$ padhta hai, ek frozen constant blob ka spread zero hota hai, aur ek wild heavy-tailed blob box ko overflow kar deta hai isliye hum uske cousin characteristic function pe switch karte hain (Step 7). Aakhir mein, do independent blobs ko glue karo aur unke boxes simply multiply hote hain — woh ek fact probability theory ke aadhe hisse ko power karta hai (Step 8).

Recall Quick self-check

Differentiate karne ke baad $t = 0$ kyun set karte hain, $t = 1$ kyun nahi? ::: $t = 0$ set karne se sirf woh shelf bachti hai jo constant pe lower ki gayi thi; $t = 1$ se baaki saari shelves zinda rahti hain aur moments aapas mein mix ho jaate hain. Kya cheez guarantee karti hai ki har $X$ ke liye $M_{X} (0) = 1$ hoga? ::: $M_{X} (0) = E [e^{0}] = E [1] = 1$ . Kaun si single assumption $M_{X + Y} = M_{X} M_{Y}$ allow karti hai? ::: Independence of $X$ aur $Y$ .