Expected value, variance, standard deviation — properties: Exercises

Two summaries of a random variable $X$ (a number spat out by a chance experiment) run through everything below:

the expected value $E [X]$ — the balance point of the probability mass;
the variance $Var (X) = E [X^{2}] - (E [X])^{2}$ — the average squared distance from that balance point; and its square root the standard deviation $σ = Var (X)$ .

The whole toolbox, on one card so you never have to scroll:

Level 1 — Recognition

Goal: spot which rule fires. No heavy algebra.

Recall Solution 1.1

(a) Expectation is linear, so the constants slide straight through: $E [3 X - 2] = 3 E [X] - 2 = 3 (4) - 2 = 10.$ (b) For variance the shift $- 2$ vanishes (sliding the whole picture left/right does not change spread) and the multiplier $3$ gets squared: $Var (3 X - 2) = 3^{2} Var (X) = 9 \cdot 9 = 81.$ (c) Standard deviation is the square root: $SD (3 X - 2) = 81 = 9$ . Equivalently $∣3∣ \cdot SD (X) = 3 \cdot 9 = 3 \cdot 3 = 9$ . ✓

Recall Solution 1.2

A 0/1 (indicator) variable has $E [X] = p$ and $Var (X) = p (1 - p)$ (see Bernoulli and Binomial Distributions). $E [X] = 0.2, Var (X) = 0.2 \cdot 0.8 = 0.16.$ Why $p (1 - p)$ ? Because for a 0/1 variable $X^{2} = X$ , so $E [X^{2}] = p$ , and $Var = E [X^{2}] - (E [X])^{2} = p - p^{2}$ .

Recall Solution 1.3

(a) False. Linearity of expectation holds always — even for dependent variables. It never needs independence. (b) True (essentially). The sum rule carries an extra term $2 Cov (X, Y)$ . Independence forces $Cov = 0$ , so the two variances simply add. (Strictly, uncorrelated is enough — but independence guarantees it.)

Level 2 — Application

Goal: crank the machine on real distributions.

Recall Solution 2.1

$E [X] = 1 (0.7) + 0 (0.3) = 0.7.$ $E [X^{2}] = 1^{2} (0.7) + 0^{2} (0.3) = 0.7 (because X^{2} = X for a 0/1 variable) .$ $Var (X) = E [X^{2}] - (E [X])^{2} = 0.7 - 0.49 = 0.21.$ $σ = 0.21 \approx 0.4583.$

Recall Solution 2.2

Center of mass (each value weighted by its probability): $E [X] = 0 (0.1) + 1 (0.2) + 2 (0.3) + 3 (0.4) = 0 + 0.2 + 0.6 + 1.2 = 2.0.$ Second moment via LOTUS with $g (x) = x^{2}$ : $E [X^{2}] = 0 (0.1) + 1 (0.2) + 4 (0.3) + 9 (0.4) = 0 + 0.2 + 1.2 + 3.6 = 5.0.$ Variance with the computational formula: $Var (X) = 5.0 - 2. 0^{2} = 5.0 - 4.0 = 1.0.$ $σ = 1.0 = 1.0.$

Recall Solution 2.3

Mean: $E [Y] = - 4 E [X] + 7 = - 4 (2.0) + 7 = - 1.$ Variance: the multiplier squares (so the sign disappears) and the $+ 7$ shift does nothing: $Var (Y) = (- 4)^{2} Var (X) = 16 \cdot 1.0 = 16.$ SD: $σ_{Y} = ∣ - 4∣ \cdot σ_{X} = 4 \cdot 1.0 = 4.0.$ The absolute value is what keeps $σ_{Y} \geq 0$ even though $a = - 4$ is negative.

The figure below shows why the sign drops out and the shift is invisible to spread.

Figure — Expected value, variance, standard deviation — properties

Level 3 — Analysis

Goal: reason about when a rule bends, and about combined variables.

Recall Solution 3.1

$X + X = 2 X$ , so by the scale rule $Var (X + X) = Var (2 X) = 2^{2} Var (X) = 4 \cdot 5 = 20.$ Naively adding variances gives $Var (X) + Var (X) = 5 + 5 = 10$ . They differ because the sum rule needs independence, and $X$ is perfectly correlated with itself. The missing piece is $2 Cov (X, X) = 2 Var (X) = 2 \cdot 5 = 10$ , and indeed $10 + 10 = 20$ . ✓

Recall Solution 3.2

Use $Var (a X + bY) = a^{2} Var (X) + b^{2} Var (Y) + 2 ab Cov (X, Y)$ . (a) $a = b = 1$ : $Var (X + Y) = 4 + 9 + 2 (1) (1) (- 3) = 13 - 6 = 7.$ (b) $a = 1, b = - 1$ : the cross term flips sign because $2 ab = 2 (1) (- 1) = - 2$ : $Var (X - Y) = 4 + 9 + 2 (1) (- 1) (- 3) = 13 + 6 = 19.$ (c) If independent, $Cov = 0$ , so both become $4 + 9 = 13$ . Note subtraction still adds variances — spreads accumulate whether you add or subtract random quantities.

Recall Solution 3.3

Independence lets variances add: $Var (S) = Var (X_{1}) + Var (X_{2}) = \frac{35}{12} + \frac{35}{12} = \frac{70}{12} = \frac{35}{6} \approx 5.833.$ For the average, first scale then use independence: $Var (\overset{ˉ}{X}) = Var (\frac{1}{2} S) = \frac{1}{4} Var (S) = \frac{1}{4} \cdot \frac{35}{6} = \frac{35}{24} \approx 1.458.$ The average is less spread than a single roll — the seed of the Law of Large Numbers: averaging many independent copies shrinks variance like $1/ n$ .

Level 4 — Synthesis

Goal: chain several rules and standardise.

Recall Solution 4.1

Write $Z = a X + b$ with $a = \frac{1}{σ} = \frac{1}{5}$ and $b = - \frac{μ}{σ} = - \frac{50}{5} = - 10$ . Mean: $E [Z] = \frac{1}{5} E [X] - 10 = \frac{1}{5} (50) - 10 = 10 - 10 = 0.$ Variance: $Var (Z) = (\frac{1}{5})^{2} Var (X) = \frac{1}{25} \cdot 25 = 1.$ So any variable can be reshaped to mean $0$ , variance $1$ — the standard form used in the Central Limit Theorem. Subtracting $μ$ centers it; dividing by $σ$ rescales the spread to exactly $1$ .

Recall Solution 4.2

Equal weights. With independence the cross term is $0$ : $Var (0.5 X + 0.5 Y) = 0. 5^{2} (4) + 0. 5^{2} (16) = 0.25 (4) + 0.25 (16) = 1 + 4 = 5.$ Optimal weight. For $W = w X + (1 - w) Y$ (independent), $Var (W) = w^{2} (4) + (1 - w)^{2} (16) .$ Differentiate and set to zero (calculus finds the bottom of this upward parabola — the flat point where slope $= 0$ ): $\frac{d}{d w} [4 w^{2} + 16 (1 - w)^{2}] = 8 w - 32 (1 - w) = 0 ⟹ 8 w - 32 + 32 w = 0 ⟹ 40 w = 32 ⟹ w = 0.8.$ Minimum variance: $Var (W) = 0. 8^{2} (4) + 0. 2^{2} (16) = 0.64 (4) + 0.04 (16) = 2.56 + 0.64 = 3.2.$ Notice the safer asset ( $Var = 4$ ) gets the larger weight $0.8$ — you lean on the calmer variable to reduce total spread.

Level 5 — Mastery

Goal: prove from the definitions; no plugging into memorised results.

Recall Solution 5.1

Let $V = a X + b$ . By linearity $E [V] = a μ + b$ where $μ = E [X]$ . Then $V - E [V] = (a X + b) - (a μ + b) = a (X - μ) .$ The $+ b$ cancels — that is why a shift cannot change spread. Now square and take expectation: $Var (V) = E [a^{2} (X - μ)^{2}] = a^{2} E [(X - μ)^{2}] = a^{2} Var (X),$ pulling the constant $a^{2}$ out by linearity. $■$

Recall Solution 5.2

Expand with linearity, treating $c$ as a constant: $M (c) = E [X^{2}] - 2 c E [X] + c^{2} .$ This is an upward parabola in $c$ . Differentiate and set to zero: $M^{'} (c) = - 2 E [X] + 2 c = 0 ⟹ c = E [X] .$ (Second derivative $M^{''} (c) = 2 > 0$ confirms a minimum.) Plug $c = μ = E [X]$ back in: $M (μ) = E [X^{2}] - 2 μ \cdot μ + μ^{2} = E [X^{2}] - μ^{2} = Var (X) .$ So the mean is the single best constant predictor of $X$ under squared error, and the unavoidable leftover error is exactly the variance. $■$

Recall Solution 5.3

By the computational formula, $E [X^{2}] - (E [X])^{2} = Var (X)$ . Variance is an average of the non-negative quantity $(X - μ)^{2}$ , hence $Var (X) \geq 0$ , which gives $E [X^{2}] \geq (E [X])^{2} .$ Equality holds iff $Var (X) = 0$ , i.e. iff $(X - μ)^{2} = 0$ with probability $1$ — meaning $X$ is a constant (no randomness at all). This is the simplest instance of Jensen's Inequality for the convex function $g (x) = x^{2}$ : averaging then squaring undershoots squaring then averaging. $■$

Recall ladder

Recall Quick self-test (hide answers)

Var(3X-2) with Var(X)=9 ::: 81 Var(X+X) with Var(X)=5 ::: 20 Var(X-Y), independent, Var(X)=4, Var(Y)=9 ::: 13 Best constant predictor c minimising E[(X-c)^2] ::: c = E[X] Divide X-mu by which quantity to make Var = 1 ::: sigma (the SD, not the variance) Value of E[X^2]-(E[X])^2 ::: Var(X), always >= 0