4.9.8Probability Theory & Statistics

Common continuous distributions — Uniform, Normal, Exponential, Gamma, Beta

1,902 words9 min readdifficulty · medium2 backlinks

1. The common machinery (derive once, reuse five times)


2. Uniform   XU(a,b)\;X\sim U(a,b)


3. Exponential   XExp(λ)\;X\sim \text{Exp}(\lambda)


4. Gamma   XGamma(α,λ)\;X\sim \text{Gamma}(\alpha,\lambda)


5. Normal   XN(μ,σ2)\;X\sim N(\mu,\sigma^2)

Figure — Common continuous distributions — Uniform, Normal, Exponential, Gamma, Beta

6. Beta   XBeta(α,β)\;X\sim \text{Beta}(\alpha,\beta)


Common mistakes (Steel-man + fix)


Recall Feynman: explain to a 12-year-old

Imagine waiting for popcorn to pop. Uniform is like "the prize is hidden somewhere on this ruler, no clue where." Exponential is "how long till the first pop" — and it doesn't matter how long you've already waited, the next pop is just as surprising. Gamma is "how long till the fifth pop" — five waits stacked together. Normal is the famous bell: if you average lots of random things (like many kids' heights), the average always makes that hill shape. Beta is a dial that only goes from 0 to 1 — great when the thing you're guessing is itself a chance, like "what fraction of free throws will I make?"


Connections

  • Central Limit Theorem — why Normal appears everywhere.
  • Poisson Process — generates both Exponential and Gamma.
  • Gamma Function and Beta Function — the integration tools.
  • Moment Generating Functions — slick way to get means/variances.
  • Conjugate Priors in Bayesian Inference — Beta is conjugate to the Binomial.
  • Discrete Distributions — Geometric is the discrete cousin of Exponential.

Flashcards

Uniform U(a,b)U(a,b) PDF
f(x)=1baf(x)=\frac{1}{b-a} for axba\le x\le b, else 0.
Uniform variance
(ba)212\frac{(b-a)^2}{12} (NOT /4).
Exponential PDF and CDF
f(t)=λeλtf(t)=\lambda e^{-\lambda t}, F(t)=1eλtF(t)=1-e^{-\lambda t}, t0t\ge0.
Exponential mean and variance
mean 1/λ1/\lambda, variance 1/λ21/\lambda^2.
Memoryless property
P(X>s+tX>s)=P(X>t)=eλtP(X>s+t\mid X>s)=P(X>t)=e^{-\lambda t}.
Relation Exp↔Gamma
Exp(λ)(\lambda) = Gamma(α=1,λ)(\alpha{=}1,\lambda); sum of kk Exp = Gamma(k,λ)(k,\lambda).
Gamma PDF
f(x)=λαΓ(α)xα1eλxf(x)=\frac{\lambda^\alpha}{\Gamma(\alpha)}x^{\alpha-1}e^{-\lambda x}, x>0x>0.
Gamma mean and variance
mean α/λ\alpha/\lambda, variance α/λ2\alpha/\lambda^2.
Normal PDF
1σ2πexp((xμ)22σ2)\frac{1}{\sigma\sqrt{2\pi}}\exp(-\frac{(x-\mu)^2}{2\sigma^2}).
Standardize a Normal
Z=(Xμ)/σN(0,1)Z=(X-\mu)/\sigma\sim N(0,1), P(Xx)=Φ((xμ)/σ)P(X\le x)=\Phi((x-\mu)/\sigma).
68-95-99.7 rule
68%\approx68\% within 1σ1\sigma, 95%95\% within 2σ2\sigma, 99.7%99.7\% within 3σ3\sigma.
Beta PDF
xα1(1x)β1B(α,β)\frac{x^{\alpha-1}(1-x)^{\beta-1}}{B(\alpha,\beta)} on (0,1)(0,1), B=Γ(α)Γ(β)Γ(α+β)B=\frac{\Gamma(\alpha)\Gamma(\beta)}{\Gamma(\alpha+\beta)}.
Beta mean and variance
mean αα+β\frac{\alpha}{\alpha+\beta}, var αβ(α+β)2(α+β+1)\frac{\alpha\beta}{(\alpha+\beta)^2(\alpha+\beta+1)}.
Beta(1,1) equals
Uniform(0,1).
Gamma function recursion
Γ(α+1)=αΓ(α)\Gamma(\alpha+1)=\alpha\Gamma(\alpha), Γ(n)=(n1)!\Gamma(n)=(n-1)!, Γ(1/2)=π\Gamma(1/2)=\sqrt\pi.
Why 2π\sqrt{2\pi} in Normal
Gaussian integral ez2/2dz=2π\int e^{-z^2/2}dz=\sqrt{2\pi} (polar-coord proof) normalizes it.

Concept Map

defines

plug f x into

plug f x into

plug f x into

integrates

normalizes

max ignorance flat density

wait for first event

sum of k gives

derived via

arises from CLT sums

models a random probability

PDF f x and CDF F x

Mean and Variance integrals

Gamma function

Poisson process rate lambda

Uniform U a,b

Normal

Exponential

Gamma

Beta on 0,1

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho beta, ye jo continuous distributions hain, inko yaad karne ka smart tarika ye hai ki inhe alag-alag isolated formulas ki tarah mat dekho — inhe ek family tree ki tarah samjho. Har distribution ek alag physical story batati hai. Jaise Uniform matlab total ignorance — pata hai value aa aur bb ke beech hai bas, isliye density flat hoti hai, har point equally likely. Normal aata hai jab bahut saare chhote-chhote effects add hote hain (Central Limit Theorem). Exponential batata hai ki pehla random event kitni der mein aayega — jaise bus kab aayegi. Gamma matlab kk-th event ka wait, aur Beta ek random probability hoti hai jo 00 se 11 ke beech rehti hai.

Ab sabse zaroori baat — ye jo saari machinery hai (PDF, CDF, mean, variance) wo ek hi baar seekhni padti hai, phir har distribution mein reuse hoti hai. Mean nikalne ke liye bas xf(x)dx\int x\,f(x)\,dx karo, aur variance ke liye E[X2]μ2\mathbb{E}[X^2]-\mu^2. Sirf f(x)f(x) badalta hai, baaki integral ka logic same rehta hai. Ye jo 80/20 rule hai na — agar tumne kisi ek distribution ke liye ye integrate karna seekh liya, to baaki chaar mein bhi wahi steps repeat honge, bas thoda algebra alag hoga. Isliye ghabrao mat, ek baar machinery pakad lo.

Ek beautiful cheez dekho — memorylessness ki. Exponential distribution ki khaasiyat ye hai ki agar tum bus ke liye 5 minute wait kar chuke ho, to bhi aage ka wait waisa hi hai jaise abhi start kiya ho. Formula mein P(X>s+tX>s)=eλtP(X>s+t \mid X>s) = e^{-\lambda t} — matlab process elapsed time ko "bhool" jata hai. Aur Gamma to bas kई Exponentials ka sum hai, isliye family tree wali baat samajh aati hai. Ye concepts real life mein bahut kaam aate hain — waiting times, reliability, sabme. Isliye stories yaad rakho, formulas apne aap flow karenge.

Go deeper — visual, from zero

Test yourself — Probability Theory & Statistics

Connections