Random variables (discrete and continuous)

1.3.5 · AI-ML › Probability & Statistics

Ek random variable ek function hota hai jo kisi random experiment ke outcomes ko real numbers pe map karta hai. Yeh messy real world (coin flips, pixel intensities, user clicks) aur clean mathematical world ke beech ka bridge hai, jahan hum probabilities aur expectations calculate kar sakte hain.

Why Random Variables Matter in ML

Machine learning mein, almost sab kuch ek random variable hai:

• Features: pixel values, word counts, sensor readings
• Labels: class assignments, regression targets
• Model parameters: training ke dauran sample kiye gaye weights
• Predictions: uncertainty ke saath outputs

RVs ko samajhne se aap:

1. Data aur predictions mein uncertainty quantify kar sakte ho
2. Aise loss functions design kar sakte ho jo aapke data distribution se match karein
3. Complex distributions se sample kar sakte ho (GANs, VAEs)
4. Model behavior ke baare mein probabilistically reason kar sakte ho

The Core Intuition

Example: Ek die roll karo. Outcome space hai {⚀, ⚁, ⚂, ⚃, ⚄, ⚅}. Define karo $X$ = "dots ki sankhya". Tab $X$ map karta hai ⚂ → 3, ⚅ → 6, wagera. $X$ random variable hai, outcome "3" khud nahi.

Discrete Random Variables

Probability Mass Function (PMF)

Probability mass function $P(X=x)$ woh probability deta hai ki $X$ exactly $x$ value leta hai.

Requirements (yeh ensure karte hain ki yeh ek valid probability distribution hai):

1. Non-negativity: $P(X=x)\ge 0$ sabhi $x$ ke liye
2. Normalization: ${\sum }_{x}P(X=x)=1$ (saari probabilities ka sum 1 hota hai)

Yeh rules kyun? Non-negativity: probabilities negative nahi ho sakti. Normalization: kuch na kuch zaroor hona chahiye, isliye total probability = 1.

First principles se:

• Socho experiment $N$ baar repeat ho raha hai, values milti hain $x_1,x_2,...,x_N$
• Sample average hai $\frac{1}{N}{\sum }_{i=1}^N{x}_i$
• Identical values group karo: agar $x_j$ $n_j$ baar aata hai, average = $\frac{1}{N}{\sum }_j{n}_j{x}_j$
• Jab $N\to \infty$, $\frac{n_j}{N}\to P(X=x_j)$ (frequency probability pe converge hoti hai)
• Isliye: $E[X]={\sum }_{x}x\cdot P(X=x)$

Har term $P(X=x)$ se weight kyun hota hai? Zyada probable values long run mein zyada baar aati hain, isliye woh average mein zyada contribute karti hain.

PMF: $P(X=1)=p,P(X=0)=1-p$

Compact form: $P(X=x)=p^x(1-p{)}^{1-x}$ for $x\in \{0,1\}$

Yeh form kyun? Jab $x=1$: $p^1(1-p{)}^0=p$. Jab $x=0$: $p^0(1-p{)}^1=1-p$. Yeh dono cases ek expression mein likhne ka ek clever tarika hai.

Expected value: $E[X]=0\cdot P(X=0)+1\cdot P(X=1)=0\cdot (1-p)+1\cdot p=p$

$E[X]=p$ kyun? Agar $p=0.3$ ke saath 100 baar flip karo, toh expect karte ho lagbhag 30 heads. Average = 30/100 = 0.3 = $p$.

ML Application: Binary classification labels (spam/not spam). Logistic regression $p$ output karta hai, jo literally $E[X]$ hai Bernoulli RV "kya yeh example class 1 hai?" ke liye.

PMF ki Derivation:

• Kisi bhi specific sequence ki probability jisme $k$ heads hain: $p^k(1-p{)}^{n-k}$ (heads $k$ baar prob $p$ ke saath aate hain, tails $n-k$ baar prob $1-p$ ke saath)
• Aise sequences ki sankhya ($n$ flips mein $k$ heads ke orderings): $\left(\genfrac{}{}{0ex}{}{n}{k}\right)=\frac{n!}{k!(n-k)!}$
• Multiplication kyun? Har specific sequence ki prob $p^k(1-p{)}^{n-k}$ hai, aur $\left(\genfrac{}{}{0ex}{}{n}{k}\right)$ mutually exclusive sequences hain

$P(X=k)=\left(\genfrac{}{}{0ex}{}{n}{k}\right)p^k(1-p{)}^{n-k},k=0,1,...,n$

Expected value: $E[X]=np$

Kyun? Har flip expected value $p$ contribute karta hai. $n$ independent flips ke saath: $E[X_1+...+X_n]=E[X_1]+...+E[X_n]=p+...+p=np$.

ML Application: Neural networks mein Dropout. Har neuron independently probability $p$ ke saath "survive" karta hai. Ek layer mein active neurons ki sankhya ~ Binomial.

Continuous Random Variables

Discrete se key difference: $P(X=x)=0$ kisi bhi specific $x$ ke liye. Kyun? Infinitely many points, probability infinitely thin spread ho jaati hai. Hume intervals ke baare mein poochna hoga: $P(a\le X\le b)$.

Probability Density Function (PDF)

Probability density function $f(x)$ ek probability NAHI hai—yeh ek density hai. Ise socho "probability per unit length" ki tarah.

Requirements:

1. Non-negativity: $f(x)\ge 0$ sabhi $x$ ke liye
2. Normalization: ${\int }_{-\infty }^{\infty }f(x)dx=1$

Integration kyun? Tum interval par infinitesimal probabilities $f(x)dx$ ko sum kar rahe ho. "$dx$" density ko probability mein convert karta hai: $f(x)$ ke units hain "probability per unit", $dx$ ke units hain "unit", product ke units hain "probability".

Geometric intuition: Probability = $f(x)$ curve ke neeche $a$ aur $b$ ke beech ka area.

$E[X]={\int }_{-\infty }^{\infty }x\cdot f(x)dx$

$x\cdot f(x)$ kyun? Discrete jaisi hi logic: har value $x$ ko uski "probability" se weight karo (yahan, density × $dx$).

PDF: $f(x)=\{\begin{array}{ll}\frac{1}{b-a}& \text{if }a\le x\le b\\ 0& \text{otherwise}\end{array}$

Yeh form kyun? $[a,b]$ mein constant density (equally likely). 1 tak integrate hona chahiye: ${\int }_a^b\frac{1}{b-a}dx=\frac{1}{b-a}\cdot (b-a)=1✓$

Expected value: $E[X]={\int }_a^{b}x\cdot \frac{1}{b-a}dx=\frac{1}{b-a}{\int }_a^{b}xdx=\frac{1}{b-a}\cdot \frac{x^2}{2}{|}_a^b=\frac{1}{b-a}\cdot \frac{b^2-a^2}{2}$

Simplify karo: $b^2-a^2=(b-a)(b+a)$, isliye: $E[X]=\frac{(b-a)(b+a)}{2(b-a)}=\frac{b+a}{2}$

Midpoint kyun? Symmetry. Dono taraf equally likely, isliye average center hota hai.

ML Application: Neural networks mein weight initialization aksar uniform distributions use karta hai (jaise Xavier/Glorot initialization $U[-\sqrt{6/(n_{in}+n_{out})},\sqrt{6/(n_{in}+n_{out})}]$ se sample karta hai).

PDF: $f(x)=\frac{1}{\sqrt{2\pi {\sigma }^2}}\exp \left(-\frac{(x-\mu {)}^2}{2{\sigma }^2}\right)$

Parameters: $\mu$ (mean, location), ${\sigma }^2$ (variance, spread)

Yeh functional form kyun? (Mean/variance constraints ke under maximum entropy se brief derivation—full proof complex hai, lekin intuition):

• Hum "sabse random" distribution chahte hain sirf mean aur variance ki knowledge diye gaye
• $E[X]=\mu$, $E[(X-\mu {)}^2]={\sigma }^2$ subject to entropy maximize karna yeh exponential-quadratic form deta hai
• $1/\sqrt{2\pi {\sigma }^2}$ ensure karta hai ki $\int f(x)dx=1$

$\exp (-x^2)$ shape kyun? Yeh probability ko mean ($\mu$) ke paas concentrate karta hai, tails quickly khatam ho jaate hain. Exponent mein quadratic smoothness aur "bell" shape ensure karta hai.

Expected value: $\mu$ ke around symmetry se, $E[X]=\mu$.

ML Applications:

1. Data mein Gaussian noise: $y=f(x)+ϵ$, jahan $ϵ\sim \mathcal{N}(0,{\sigma }^2)$
2. Weight initialization (jaise He initialization: $\mathcal{N}(0,2/n_{in})$)
3. Gaussian processes, variational autoencoders (VAE latent space)
4. Linear regression assume karta hai ${ϵ}_i\sim \mathcal{N}(0,{\sigma }^2)$

Cumulative Distribution Function (CDF)

Discrete aur continuous dono RVs ka CDF hota hai $F(x)=P(X\le x)$.

Discrete RV ke liye: $F(x)={\sum }_{x_i\le x}P(X=x_i)$ (PMF values ko sum karo)

Continuous RV ke liye: $F(x)={\int }_{-\infty }^{x}f(t)dt$ (PDF integrate karo)

Useful kyun?

• CDFs hamesha exist karte hain (PMFs/PDFs mixed distributions ke liye nahi hote)
• Probabilities compute karna aasaan: $P(a<X\le b)=F(b)-F(a)$
• PDF se relationship: $f(x)=\frac{dF(x)}{dx}$ (PDF, CDF ka derivative hai)

ML Application: Percentile-based metrics, quantile regression, inverse transform sampling (kisi bhi distribution se samples generate karo uniform random numbers par $F^{-1}$ apply karke).

Common Mistakes

Kyun sahi lagta hai: Hum discrete RVs se $P(X=x)$ ke aadat hain.

Fix: $f(x)$ density hai, probability nahi. Units: $f(x)$ "probability per unit" hai. $f(x)=2$ matlab "locally, probability 2 units per unit length ki speed se accumulate ho rahi hai". Actual probability: $P(X\in [x,x+dx])=f(x)\cdot dx$ (jo chhote $dx$ ke liye chhota hai).

Check: $X\sim U[0,0.3]$ ke liye, $f(x)=1/0.3\approx 3.33>1$. Lekin $P(0\le X\le 0.3)=3.33\times 0.3=1$ ✓

Kyun sahi lagta hai: Dono weighted averages use karte hain.

Fix:

• Discrete: E_X = sum(x * P(x) for x in values)
• Continuous: E_X = integrate(x * f(x), x, -inf, inf) ya E_X = np.mean(samples) samples ke liye

NumPy mein, agar tumhare paas continuous RV ke samples hain, toh np.mean(samples) use karo, na ki sum(x * count(x)) (jo discrete assume karta hai).

Kyun sahi lagta hai: Hum thermometers par "20°C" measure karte hain.

Fix: Real measurements finite precision hoti hai. Tumhara thermometer "20°C" padhta hai = "19.5 aur 20.5°C ke beech" (ek interval). Continuous RVs ke liye, hamesha intervals ke baare mein poochho: $P(19.5\le X\le 20.5)$.

Active Recall

Ek random variable us number ka nickname jaisa hai. Hum ise $X$ kehte hain. Agar hum die roll karein, $X$ 1, 2, 3, 4, 5, ya 6 ho sakta hai (sirf yahi numbers, beech mein kuch nahi). Yeh discrete hai—jaise candies ginana.

Lekin agar hum ek wheel spin karein jo 0 se 100 ke beech kahin bhi land kar sakta hai, $X$ 23.7145... ya 99.001... ya beech mein koi bhi number ho sakta hai. Yeh continuous hai—jaise cup mein kitna paani hai measure karna.

Discrete ke liye, hum keh sakte hain "exactly 3 milne ki probability hai..." (jaise fair die ke liye $P(X=3)=1/6$). Continuous ke liye, hum "exactly 23.7145" nahi keh sakte kyunki infinite possibilities hain—yeh aise hai jaise beach par ek specific ret ka daana dhundna. Isliye hum poochte hain "X ke 20 aur 30 ke beech hone ki probability kya hai?" aur us range mein probabilities add karte hain (integration).

Cool part yeh hai: AI mein almost sab kuch ek random variable hai—image mein kaunsa digit hai, agla word kya aayega, email spam hai ya nahi. RVs samajhna matlab AI uncertainty kaise handle karta hai yeh samajhna hai.

Memory Aids

• Discrete = Distinct, Discrete math (integers)
• PMF = Probability Mass (mass points par baith jaati hai)
• PDF = Probability Density (fluid ki tarah spread)

Alphabet trick:

• PMF Sum use karta hai ($\Sigma$)
• PDF Integral use karta hai ($\int$)
• "S comes before I" → discrete ke liye sum use karo, continuous ke liye integral

Connections

• Probability Space – random variables sample spaces par measurable functions hote hain
• Expectation and Variance – random variables ke moments, higher-order statistics
• Joint and Conditional Distributions – multivariate RVs, dependence structure
• Central Limit Theorem – sums Gaussian pe converge kyun karte hain
• Maximum Likelihood Estimation – data pe distributions fit karna ($p$, $\mu$, $\sigma$ find karna)
• Bernoulli and Categorical Distributions – classification labels discrete RVs ki tarah
• Gaussian Distribution – regression noise, VAE latents ke liye continuous RV
• Sampling Methods – RV samples generate karna (inverse transform, rejection sampling)
• Information Theory – RV distributions ki entropy

#flashcards/ai-ml

Random variable kya hota hai? :: Ek function jo random experiment ke outcomes ko real numbers pe map karta hai. Yeh real-world randomness aur mathematical probability ke beech ka bridge hai.

Continuous RV ke liye expected value formula? :: $E[X]={\int }_{-\infty }^{\infty }x\cdot f(x)dx$. Density par weighted integral.

Concept Map