Expected value, variance, standard deviation — properties

4.9.4 · Maths › Probability Theory & Statistics

1. Expected Value — WHAT, WHY, HOW

WHY yeh formula? Probability ko mass ki tarah socho. Number line par position $x_{i}$ pe mass $p_{i}$ rakh do. Us mass distribution ka balance point (center of mass) exactly $\sum x_{i} p_{i}$ hota hai. Isliye $E [X]$ ko mean kehte hain — yeh woh point hai jahan distribution ek knife edge par balance karega.

HOW hum isse use karte hain (sabse important tool): Law of the Unconscious Statistician (LOTUS). Kisi function of $X$ ka average lene ke liye, tumhe $g (X)$ ki distribution nahi chahiye: $E [g (X)] = \sum_{i} g (x_{i}) p_{i} or \int g (x) f (x) d x$

Linearity of expectation (the workhorse)

$E [a X + b]$ ki derivation scratch se (discrete): $E [a X + b] = \sum_{i} (a x_{i} + b) p_{i} = a \sum_{i} x_{i} p_{i} + b \sum_{i} p_{i} = a E [X] + b \cdot 1.$ Har step kyun? Pehle humne LOTUS use kiya $g (x) = a x + b$ ke saath. Phir sum ko split kiya (sums linear hote hain). Phir constants bahar nikale, aur $\sum_{i} p_{i} = 1$ use kiya (probabilities ka sum ek hota hai). Done.

2. Variance — spread measure karna

WHY squared, sirf $∣ X - μ ∣$ kyun nahi? Agar hum directly $X - μ$ average karte toh $0$ milta (positives, negatives ko cancel karte hain — literally yahi mean hoti hai). Squaring se sab kuch positive ho jaata hai, bade deviations zyada punish hote hain, aur algebra clean rehta hai (yeh differentiable hai, unlike $∣ \cdot ∣$ ).

First principles se derivation:

Var (X) = E [(X - μ)^{2}] = E [X^{2} - 2 μ X + μ^{2}]

= E [X^{2}] - 2 μ E [X] + μ^{2} (linearity)

= E [X^{2}] - 2 μ \cdot μ + μ^{2} = E [X^{2}] - μ^{2} .

Har step kyun? Square expand karo (algebra). Linearity apply karo — note karo $μ$ ek constant hai, isliye bahar aa jaata hai. Phir $E [X] = μ$ substitute karo. Beech ke do terms $- μ^{2}$ mein collapse ho jaate hain.

Standard deviation

3. How Variance behaves under scaling & shifting

Derivation. Maano $Y = a X + b$ . Tab $E [Y] = a μ + b$ . To

Var (Y) = E [(Y - E [Y])^{2}] = E [(a X + b - a μ - b)^{2}] = E [a^{2} (X - μ)^{2}] = a^{2} Var (X) .

Kyun? $+ b$ cancel ho jaata hai — poori distribution ko slide karna uska spread nahi badalta. $a$ factor out ho jaata hai aur square ho jaata hai kyunki variance squared units mein hoti hai. Root lene par $∣ a ∣$ milta hai (absolute value, kyunki SD ≥ 0).

Figure — Expected value, variance, standard deviation — properties

4. Variance of a sum (jahan independence matter karta hai)

Derivation sketch. Maano $μ_{X}, μ_{Y}$ means hain.

Var (X + Y) = E [((X - μ_{X}) + (Y - μ_{Y}))^{2}]

= E [(X - μ_{X})^{2}] + E [(Y - μ_{Y})^{2}] + 2 E [(X - μ_{X}) (Y - μ_{Y})] .

Pehle do terms variances hain; aakhri term $2 Cov (X, Y)$ hai. Independence isse kyun khatam karta hai? Kyunki independent variables ke liye $E [X Y] = E [X] E [Y]$ , jisse covariance $0$ ho jaata hai.

5. Worked examples

6. Common mistakes (Steel-manned)

7. Active recall

Recall Self-test (hide karke answer karo)

Variance ki computational formula batao aur derive karo.
$Var (a X + b)$ kya hai aur $b$ kyun vanish ho jaata hai?
$Var (X + Y) = Var (X) + Var (Y)$ kab hota hai?
SD lene ke liye hum square root kyun lete hain?
Kya linearity of expectation dependent variables ke liye bhi true hai?

Recall Feynman: ek 12-saal ke bachche ko explain karo

Socho tum ek number line par darts phenko. Jis average jagah tum hit karte ho woh expected value hai — tumhare cluster ka middle. Variance poochta hai: mere darts us middle ke aas-paas kitne scattered hain? Hum har dart ki middle se distance measure karte hain, use square karte hain (taaki left aur right dono misses "bure" count hon), aur unka average lete hain. Standard deviation wahi scatter hai jo normal distance units mein measure hoti hai. Agar tum poora target sideways slide karo, tumhara scatter nahi badlta — lekin agar tum target picture ko twice as wide zoom karo, tumhara scatter double ho jaata hai (aur squared scatter, yaani variance, four-times ho jaata hai).

Flashcards

What is the definition of expected value for a discrete RV?

E [X] = \sum_{i} x_{i} p_{i}

— probability-weighted average (center of mass).

State the computational formula for variance.

Var (X) = E [X^{2}] - (E [X])^{2}

Why do we square deviations in variance?

+

aur

-

deviations ka cancellation rokne ke liye (woh

0

sum karte) aur clean differentiable algebra ke liye.

What is

Var (a X + b)

a^{2} Var (X)

—

b

cancel ho jaata hai (shift),

a

square ho jaata hai.

What is

SD (a X + b)

∣ a ∣ SD (X)

E [X + Y] = E [X] + E [Y]

always true?

Haan — linearity of expectation dependent variables ke liye bhi hold karta hai.

When does

Var (X + Y) = Var (X) + Var (Y)

Jab

Cov (X, Y) = 0

ho (e.g.

X, Y

independent).

General formula for

Var (X + Y)

Var (X) + Var (Y) + 2 Cov (X, Y)

Define covariance.

Cov (X, Y) = E [X Y] - E [X] E [Y]

Variance of a fair die?

\frac{91}{6} - 3. 5^{2} = \frac{35}{12} \approx 2.917

Variance of a Bernoulli(

p

p (1 - p)

p = 0.5

par maximum.

Why is

E [X^{2}] \geq (E [X])^{2}

Unka difference

Var (X) \geq 0

hai (Jensen's inequality for convex

x^{2}

What is LOTUS?

E [g (X)] = \sum_{i} g (x_{i}) p_{i}

—

g (X)

ki distribution nikale bina

X

ke function ka average lo.

Connections

Probability Distributions
Covariance and Correlation
Jensen's Inequality
Law of Large Numbers — sample mean → $E [X]$
Central Limit Theorem — sums aur unki variances
Moment Generating Functions
Bernoulli and Binomial Distributions