Expectation, variance, and standard deviation

1.3.8 · AI-ML › Probability & Statistics

Overview

Ye teen concepts ek random variable ke baare mein alag-alag sawaalon ka jawaab dete hain:

Expectation (Mean): Distribution ka "center" kahan hai?
Variance: Values kitni pheli hui hain?
Standard Deviation: Kitni pheli hui hain, lekin original data ke same units mein?

Ye teeno ML mein data distributions samajhne, models compare karne, aur uncertainty quantify karne ki foundation hain.

Imagine karo aap ek experiment $N$ baar run karte ho. Value $x_{i}$ , $n_{i}$ baar aati hai. Sample average hai: $Average = \frac{1}{N} \sum_{i} n_{i} \cdot x_{i} = \sum_{i} x_{i} \cdot \frac{n _{i}}{N}$

Jab $N \to \infty$ , relative frequency $\frac{n _{i}}{N} \to P (X = x_{i})$ (law of large numbers). To: $lim_{N \to \infty} Average = \sum_{i} x_{i} \cdot P (X = x_{i}) = E [X]$

Continuous variables ke liye: Hum real line ko $d x$ width ke chhote bins mein divide karte hain. $x$ ke aas-paas wale bin mein girne ki probability $f (x) d x$ hai. Saare bins pe sum karne se integral definition milti hai.

Linearity kyun? Sum expand karo: $E [a X + bY] = \sum_{i} \sum_{j} (a x_{i} + b y_{j}) P (X = x_{i}, Y = y_{j})$ $= a \sum_{i} x_{i} P (X = x_{i}) + b \sum_{j} y_{j} P (Y = y_{j}) = a E [X] + b E [Y]$

Variance

Derivation: Do Forms

Form 1 (definition): Mean se average squared deviation. $Var (X) = E [(X - μ)^{2}]$

Form 2 (computational): Square expand karo. $Var (X) = E [X^{2} - 2 μ X + μ^{2}]$ $= E [X^{2}] - 2 μ E [X] + μ^{2}$

Ye step kyun? Expectation ki linearity use karo. $E [X] = μ$ aur $μ^{2}$ constant hai.

$= E [X^{2}] - 2 μ^{2} + μ^{2} = E [X^{2}] - μ^{2}$

Form 2 kyun? Compute karna aasaan hai— $E [X]$ aur $E [X^{2}]$ alag-alag calculate karo, phir subtract karo.

Scaling mein $a^{2}$ kyun? $Var (a X) = E [(a X)^{2}] - (E [a X])^{2} = a^{2} E [X^{2}] - a^{2} (E [X])^{2} = a^{2} Var (X)$

Shift invariant kyun? Constant add karne se distribution shift hoti hai lekin spread nahi badlta. $Var (X + b) = E [(X + b - E [X + b])^{2}] = E [(X - E [X])^{2}] = Var (X)$

Standard Deviation

Square root kyun lete hain? Original scale pe waapas aane ke liye. Agar aap heights cm mein measure karte ho, toh variance cm² mein hai, lekin SD cm mein hai.

Common Mistakes

ML Mein Ye Kyun Matter Karta Hai

Model Evaluation: Variance prediction uncertainty measure karta hai. High variance = model inconsistent hai.
Bias-Variance Tradeoff: Systematic error (bias) aur prediction spread (variance) ko balance karna.
Gradient Descent: Gradient estimates ki variance learning stability ko affect karti hai (dekho: SGD, mini-batch size).
Feature Scaling: Standardization features normalize karne ke liye mean aur SD use karta hai: $z = \frac{x - μ}{σ}$ .
Probabilistic Models: Gaussian distributions $μ$ aur $σ^{2}$ se parameterized hoti hain.
Loss Functions: MSE (Mean Squared Error) essentially prediction errors ki variance hai.

Recall Ek 12-Saal-Ke Bacche Ko Samjhao

Imagine karo aap aur aapke dost ek board pe darts phenk rahe ho.

Expectation woh hai jahan aap aim karte ho—bullseye. Agar sabne 100 baar phenko, toh expectation woh average jagah hai jahan saare darts gire.

Variance yeh hai ki aapke throws kitne bikhre hue hain. Agar saare darts bullseye ke paas tight cluster mein hain, toh variance small hai. Agar woh idhar-udhar bikhre hain, toh variance badi hai. Hum ise measure karte hain har dart ki average jagah se door dekhke, un distances ko square karke (taaki left aur right cancel na ho), aur unka average nikalke.

Standard deviation variance jaisa hi hai, lekin hum end mein square root lete hain taaki same units mein ho—jaise "on average, darts bullseye se 5 cm door girte hain" instead of "25 cm²."

AI mein, hum in cheezon ka use karte hain yeh jaanne ke liye ki koi model consistent hai (low variance) ya idhar-udhar bhatakta hai (high variance).

Connections

1.3.01-Randomvariables-and-probability-distributions — Expectation PDF/PMF pe integrate karta hai
1.3.09-Covariance-and-correlation — Variance ko variable ke pairs tak generalize karta hai
1.3.12-Central-limit-theorem — Sample mean ki variance $σ^{2} / n$ hoti hai
2.1.05-Gradient-descent-variants — Stochastic gradients ki variance convergence affect karti hai
3.2.04-Bias-variance-tradeoff — Prediction error decompose karna
4.1.02-Feature-scaling-and-normalization — Standardization $μ$ aur $σ$ use karta hai

#flashcards/ai-ml

Ek discrete random variable ki expectation kya hoti hai? :: $E [X] = \sum_{i} x_{i} \cdot P (X = x_{i})$ , saari possible values ka probability-weighted average.

Variance kya measure karta hai?

Values ki mean se average squared distance; spread ya dispersion quantify karta hai.

Variance ka computational formula kya hai?

Var (X) = E [X^{2}] - (E [X])^{2}

Constant multiplier ke saath variance kaise scale hoti hai?

Var (a X) = a^{2} Var (X)

(

a

se scale karne par variance

a^{2}

se scale hoti hai).

Standard deviation kya hai?

Variance ka square root,

σ = Var (X)

, jo spread original units mein deta hai.

Independent RVs ke liye variances kaise combine hoti hain?

Var (X + Y) = Var (X) + Var (Y)

(variances add hoti hain).

Variance mein deviations square kyun karte hain?

Positive aur negative deviations ko cancel hone se rokne ke liye, aur

E [X^{2}] - μ^{2}

jaisi algebraic properties ke liye.

Kya expectation linearity follow karta hai?

Haan,

E [a X + bY] = a E [X] + b E [Y]

kisi bhi constants

a, b

ke liye.

Ek constant ki variance kya hoti hai?

Zero,

Var (c) = 0

, kyunki constant mein koi spread nahi hoti.

Agar

E [X] = 5

aur

E [X^{2}] = 30

hai, toh

Var (X)

kya hai?

Var (X) = 30 - 5^{2} = 30 - 25 = 5