Uniform, Exponential, and Beta distributions

1.3.12 · AI-ML › Probability & Statistics

The Uniform Distribution

Derivation from First Principles

WHY uniform? Hum "no preference" chahte hain—probability density $[a, b]$ ke across constant honi chahiye aur baaki jagah zero.

WHAT does that mean mathematically? Maano constant density $c$ hai. Chunki probability ka integrate hokar 1 aana zaroori hai:

$\int_{a}^{b} c d x = 1$

HOW do we solve?

$c \cdot (b - a) = 1 ⟹ c = \frac{1}{b - a}$

Key Properties (Derived)

Expected Value:

$E [X] = \int_{a}^{b} x \cdot \frac{1}{b - a} d x = \frac{1}{b - a} \cdot \frac{x ^{2}}{2}_{a}^{b} = \frac{1}{b - a} \cdot \frac{b ^{2} - a ^{2}}{2} = \frac{b + a}{2}$

WHY? Mean bilkul midpoint hai—ek symmetric distribution ke liye yeh samajh mein aata hai!

Variance:

$E [X^{2}] = \int_{a}^{b} x^{2} \cdot \frac{1}{b - a} d x = \frac{1}{b - a} \cdot \frac{x ^{3}}{3}_{a}^{b} = \frac{b ^{3} - a ^{3}}{3 ( b - a )} = \frac{a ^{2} + ab + b ^{2}}{3}$

$Var (X) = E [X^{2}] - (E [X])^{2} = \frac{a ^{2} + ab + b ^{2}}{3} - \frac{( a + b ) ^{2}}{4} = \frac{( b - a ) ^{2}}{12}$

WHY this matters? Variance range ke square ke saath badhti hai—wider intervals matlab zyada uncertainty.

Figure — Uniform, Exponential, and Beta distributions

The Exponential Distribution

Derivation from Poisson Process

WHY exponential? Poisson process mein, events independent hote hain aur constant rate $λ$ par hote hain. Kya probability hai ki time $t$ mein koi event NA ho?

Poisson distribution se: $P (0 events in time t) = e^{- λ t}$

WHAT is the CDF? Probability ki pehla event time $t$ se PEHLE aaye:

$F (t) = P (T \leq t) = 1 - P (T > t) = 1 - e^{- λ t}$

HOW do we get the PDF? CDF ko differentiate karo:

$f (t) = \frac{d}{d t} F (t) = \frac{d}{d t} (1 - e^{- λ t}) = λ e^{- λ t}$

The Memoryless Property

CLAIM: $P (T > s + t ∣ T > s) = P (T > t)$

WHY is this profound? "Agar aap $s$ time units wait kar chuke ho bina kisi event ke, toh agle $t$ time tak wait karne ki probability waisi hi hai jaise aapne abhi shuru kiya ho." Process apna past "bhool" jaata hai.

Proof:

$P (T > s + t ∣ T > s) = \frac{P ( T > s + t )}{P ( T > s )} = \frac{e ^{- λ (s + t)}}{e ^{- λ s}} = e^{- λ t} = P (T > t)$

WHERE does this matter? Exponential AKELA continuous distribution hai jisme yeh property hoti hai. Radioactive decay, server requests ke beech time, ya constant hazard rate wale components ki failure model karta hai.

Deriving the Mean:

$E [T] = \int_{0}^{\infty} t \cdot λ e^{- λ t} d t$

Integration by parts use karo: $u = t$ , $d v = λ e^{- λ t} d t$

$= [- t e^{- λ t}]_{0}^{\infty} + \int_{0}^{\infty} e^{- λ t} d t = 0 + \frac{1}{λ} = \frac{1}{λ}$

WHY this makes sense? Agar events rate $λ = 5$ per hour par hote hain, toh events ke beech average time $\frac{1}{5}$ hour = 12 minutes hoga.

The Beta Distribution

Derivation from the Conjugate Prior Perspective

WHY does Beta exist? Maano hum ek coin flip kar rahe hain jiska bias $p$ unknown hai. $α - 1$ heads aur $β - 1$ tails observe karne ke baad, $p$ ke baare mein humara belief kaunsi distribution describe karta hai?

WHAT form should it take? Hum chahte hain:

$[0, 1]$ par support (kyunki $p$ ek probability hai)
Different beliefs represent karne ki flexibility (uniform ignorance, strong certainty, etc.)
Mathematical convenience (Binomial ka conjugate)

Beta function distribution ko normalize karta hai:

$B (α, β) = \int_{0}^{1} x^{α - 1} (1 - x)^{β - 1} d x = \frac{Γ ( α ) Γ ( β )}{Γ ( α + β )}$

jahan $Γ (n) = (n - 1)!$ integers ke liye.

Understanding Shape via Parameters

HOW do $α$ aur $β$ shape control karte hain?

$α = β = 1$ : $Beta (1, 1) = Uniform (0, 1)$ — poori ignorance
$α > β$ : Distribution 1 ki taraf skewed (zyada successes observed)
$α < β$ : Distribution 0 ki taraf skewed (zyada failures observed)
$α, β > 1$ : Unimodal (single peak)
$α, β < 1$ : U-shaped (bimodal, extremes ke paas certainty)
Large $α, β$ : Narrow distribution (high confidence)

Deriving the Mean:

$E [X] = \int_{0}^{1} x \cdot \frac{x ^{α - 1} ( 1 - x ) ^{β - 1}}{B ( α , β )} d x = \frac{1}{B ( α , β )} \int_{0}^{1} x^{α} (1 - x)^{β - 1} d x$

Integral $B (α + 1, β)$ hai:

$= \frac{B ( α + 1 , β )}{B ( α , β )} = \frac{Γ ( α + 1 ) Γ ( β ) /Γ ( α + β + 1 )}{Γ ( α ) Γ ( β ) /Γ ( α + β )}$

$= \frac{α \cdot Γ ( α )}{( α + β ) \cdot Γ ( α + β )} = \frac{α}{α + β}$

WHY this makes sense? Agar aap $α - 1$ heads aur $β - 1$ tails observe karo, toh aap expect karte ho ki true probability $\frac{α - 1}{α + β - 2}$ ke paas hogi, jo large counts ke liye $\frac{α}{α + β}$ ke kareeb aa jaati hai.

Worked example Beta for Thompson Sampling

Problem: 3 slot machines wala multi-armed bandit. Har ek ki win probability unknown hai. Exploration aur exploitation balance karne ke liye Thompson Sampling use karo. Setup:

Machine A: $Beta (5, 5)$ — 8 baar try kiya, 4 wins
Machine B: $Beta (10, 2)$ — 10 baar try kiya, 9 wins
Machine C: $Beta (2, 8)$ — 8 baar try kiya, 1 win

Algorithm: Har round mein, sample karo $p_{A} \sim Beta (5, 5)$ , $p_{B} \sim Beta (10, 2)$ , $p_{C} \sim Beta (2, 8)$ . Highest sample wala arm pull karo.

WHY this works?

Machine B ka high mean hai ( $\frac{10}{12} \approx 0.83$ ) → aksar sabse zyada sample hota hai → exploit hota hai
Lekin kabhi kabhi A ya C zyada sample karte hain → exploration naturally hoti hai
Zyada uncertainty (chhota $α + β$ ) → zyada variable samples → zyada exploration

HOW does this beat epsilon-greedy? Thompson Sampling automatically exploration adapt karta hai—uncertain arms zyada try hote hain, confident arms zyada exploit hote hain.

Connections Between the Three

Property	Uniform $(a, b)$	Exponential $(λ)$	Beta $(α, β)$
Support	$[a, b]$	$[0, \infty)$	$[0, 1]$
Use case	Koi prior info nahi	Waiting times	Probabilities
Memoryless?	Nahi	Haan	Nahi
Shape	Constant	Decreasing	Flexible
Special cases	Uniform $(0, 1)$ = Beta $(1, 1)$	Exp $(1)$ = Gamma $(1, 1)$	Beta $(1, 1)$ = Uniform

Recall Feynman Explanation (ek 12-saal ke bachche ke liye)

Socho tumhare paas teen jaadui paasey hain: The Flat Die (Uniform): Har number ka bilkul same chance hai. Bilkul ek perfectly fair spinner ki tarah—koi cheating nahi, har jagah equally likely. Hum isko tab use karte hain jab hamare paas zero information ho aur hum completely fair rehna chahte hon.

The Forgetful Die (Exponential): Yeh waiting model karta hai. Jaise next bus ka intezaar karna. Ajeeb baat? Koi farak nahi padta kitna intezaar kiya—agli minute mein bus aane ka chance HAMESHA same rehta hai. Yeh aise hai jaise universe ko amnesia hai! Yeh radioactive atoms ke decay hone ya customers ke randomly store par aane jaisi chezon ke liye kaam karta hai.

The Belief Die (Beta): Yeh probabilities guess karne ke baare mein hai. Socho tum figure out karne ki koshish kar rahe ho ki ek coin fair hai ya nahi. Tum isey kuch baar flip karte ho. Beta distribution describe karta hai ki tum kitne confident ho ki true "heads probability" kya hai. Zyada data milta hai? Beta update hoti hai, zyada confident aur accurate hoti jaati hai. Yeh aise hai jaise tumhara brain har flip ke saath smarter hota jaata hai!

Connections

Continuous Random Variables and PDFs
Bayesian Inference and Conjugate Priors
Maximum Likelihood Estimation
Poisson Process
Thompson Sampling
Xavier and He Initialization
Monte Carlo Methods

#flashcards/ai-ml

Uniform, Exponential, aur Beta distributions ke support (valid ranges) kya hain? :: Uniform: $[a, b]$ (finite interval), Exponential: $[0, \infty)$ (non-negative reals), Beta: $[0, 1]$ (unit interval)

Uniform

(a, b)

ka PDF kya hai?

f (x) = \frac{1}{b - a}

for

a \leq x \leq b

, zero otherwise. Constant ensure karta hai total probability = 1.

Uniform $(a, b)$ ka mean aur variance kya hai? :: Mean: $\frac{a + b}{2}$ (midpoint), Variance: $\frac{( b - a ) ^{2}}{12}$ (range ke square ke saath badhta hai)

Exponential distribution ki memoryless property kya hai?

P (T > s + t ∣ T > s) = P (T > t)

— agle

t

time tak wait karne ki probability independent hai ki aap kitna pehle se wait kar rahe ho.

Exponential

(λ)

ka PDF kya hai?

f (t) = λ e^{- λ t}

for

t \geq 0

, jahan

λ

rate parameter hai (events per unit time).

Exponential mein rate

λ

aur mean waiting time ka kya relation hai?

Mean

E [T] = \frac{1}{λ}

. Zyada rate → chhota average wait. Agar

λ = 4

events/hour, average wait

\frac{1}{4}

hour = 15 minutes hai.

Exponential

(λ)

ka CDF kya hai?

F (t) = 1 - e^{- λ t}

for

t \geq 0

. Time

t

ke andar event hone ki probability find karne ke liye use hota hai.

Beta

(α, β)

ka PDF kya hai?

f (x) = \frac{1}{B ( α , β )} x^{α - 1} (1 - x)^{β - 1}

for

x \in [0, 1]

, jahan

B (α, β) = \frac{Γ ( α ) Γ ( β )}{Γ ( α + β )}

normalizing Beta function hai.

Beta

(α, β)

ka mean kya hai?

E [X] = \frac{α}{α + β}

. Agar

α = 5, β = 3

, mean =

\frac{5}{8} = 0.625

Beta

(α, β)

naye Binomial data ke saath kaise update hota hai?

Agar prior Beta

(α, β)

hai aur

n

trials mein

k

successes observe hote hain, toh posterior hai Beta

(α + k, β + n - k)

. Yahi conjugacy hai!

Beta

(1, 1)

kiske barabar hai?

Uniform

(0, 1)

. Yeh ek probability ke baare mein poori ignorance represent karta hai—

[0, 1]

mein har value equally likely.

Jab dono parameters large hon toh Beta

(α, β)

ki shape kya hoti hai?

Distribution zyada concentrated ho jaati hai (lower variance), high confidence represent karta hai. Example: Beta

(100, 100)

0.5 par tightly peaked hoti hai.

Beta

(α, β)

mein

α > β

kya indicate karta hai?

Distribution 1 ki taraf skewed hoti hai (right-skewed), matlab successes ke failures se zyada evidence hai. Mean

> 0.5

Simulation mein Uniform

(0, 1)

kyun important hai?

Zyaadatar random number generators Uniform

(0, 1)

samples produce karte hain. Inhe inverse transform sampling se KISI BHI doosri distribution mein transform kiya ja sakta hai:

X = F^{- 1} (U)

jahan

U \sim Uniform (0, 1)

Beta

(α, β)

ka mode kya hai jab

α, β > 1

ho?

Mode

= \frac{α - 1}{α + β - 2}

. Yeh distribution ka peak hai, sabse likely value.

Exponential akela memoryless continuous distribution kyun hai?

Memoryless property

P (X > s + t ∣ X > s) = P (X > t)

ek functional equation ke zariye exponential distribution ko uniquely characterize karti hai jo form

e^{- λ t}

force karta hai.

ML mein initialization ke liye Uniform kaun si application use karta hai?

Xavier/Glorot initialization Uniform

(- \frac{6}{n _{in} + n _{o u t}}, \frac{6}{n _{in} + n _{o u t}})

use karta hai neural network weights initialize karne ke liye, gradient variance stable rakhne ke liye.

Thompson Sampling kya hai aur Beta kaise help karta hai?

Thompson Sampling multi-armed bandits ke liye: har arm

i

ke liye Beta

(α_{i}, β_{i})

maintain karo, har Beta se sample lo, highest sample wala arm pull karo. Beta naturally exploration (uncertain arms zyada vary karte hain) aur exploitation (high-mean arms favored hoti hain) balance karta hai.

Uniform, Exponential, and Beta distributions

Core Intuition

The Uniform Distribution

Derivation from First Principles

Key Properties (Derived)

The Exponential Distribution

Derivation from Poisson Process

The Memoryless Property

The Beta Distribution

Derivation from the Conjugate Prior Perspective

Understanding Shape via Parameters

Connections Between the Three

Connections

Concept Map