Expected value, variance, standard deviation — properties: Question bank

Definition Symbols used on this page

== $X, Y$ == are random variables — numbers that come out of a chance experiment.
== $E [X]$ is the expected value (mean)==, the probability-weighted average of $X$ ; we abbreviate it $μ$ (mu). So $μ = E [X]$ is the balance point of the distribution.
== $Var (X) = E [(X - μ)^{2}]$ is the variance==, the average squared distance from the mean.
== $SD (X) = Var (X)$ is the standard deviation==, abbreviated $σ$ (sigma); it is the spread measured back in $X$ 's own units, so $σ^{2} = Var (X)$ .
== $Cov (X, Y) = E [(X - E [X]) (Y - E [Y])] = E [X Y] - E [X] E [Y]$ is the covariance==, measuring how $X$ and $Y$ move together (see Covariance and Correlation).
== $X_{1}, \dots, X_{n}$ are $n$ independent copies of $X$ (same distribution, unlinked outcomes), and the overbar $\overset{ˉ}{X} = \frac{1}{n} (X_{1} + \dots + X_{n})$ is the sample mean==, their average. Here $n$ is the sample size.
== $p$ is a probability== (a number between $0$ and $1$ ), used below for a Bernoulli $0/1$ variable.

Every symbol on this page is one of these.

Three pictures to hold in your head

Before the traps, three quick visuals — most concept errors on this page dissolve once you see these.

Figure 1 — Why squaring stops the cancellation. Deviations $X - μ$ point left (negative) and right (positive); averaged raw, they cancel to exactly $0$ . Squaring folds both sides upward so nothing cancels.

Figure — Expected value, variance, standard deviation — properties

Figure 2 — Why variances add "in quadrature." Two independent spreads combine like the two legs of a right triangle: the total SD is the hypotenuse $σ_{X}^{2} + σ_{Y}^{2}$ , always shorter than $σ_{X} + σ_{Y}$ .

Figure 3 — Why $E [X^{2}] \geq (E [X])^{2}$ (Jensen). For the convex cup $y = x^{2}$ , the average of the curve heights sits above the curve at the average input. That vertical gap is exactly the variance.

True or false — justify

E [X + Y] = E [X] + E [Y]

requires

X

and

Y

independent.

False — linearity of expectation holds always, even for dependent variables, because expectation is just a weighted sum and sums split unconditionally.

Var (X + Y) = Var (X) + Var (Y)

requires independence.

It needs only

Cov (X, Y) = 0

(uncorrelatedness); independence is sufficient but slightly stronger than necessary. See Covariance and Correlation.

Variance can be negative if

X

takes negative values.

False — variance is the average of squared deviations, and squares are never negative, so

Var (X) \geq 0

regardless of the sign of

X

Var (X) = 0

is possible for a genuine random variable.

True, but only if

X

is a constant (a "degenerate" distribution): zero spread means every outcome equals the mean

μ

with probability one.

E [X^{2}] \geq (E [X])^{2}

for every random variable.

True — their difference is exactly

Var (X) \geq 0

, which is also Jensen's inequality applied to the convex function

x^{2}

Standard deviation is measured in the same units as

X

True — that is the whole reason we take the square root; variance lives in units-squared, and

returns us to the original scale.

SD (a X + b) = a SD (X)

False — the correct factor is

∣ a ∣

, the absolute value, because standard deviation is a distance and must stay

\geq 0

even when

a < 0

Y = - X

then

Var (Y) = Var (X)

True — using the scale rule with

a = - 1

gives

(- 1)^{2} Var (X) = Var (X)

; flipping the number line reflects the cluster but doesn't change how scattered it is.

E [X Y] = E [X] E [Y]

for all random variables.

False — this factorisation holds only when

X

and

Y

are uncorrelated (in particular, independent); otherwise the gap is exactly

Cov (X, Y)

The mean

E [X]

must be one of the values

X

can actually take.

False — a fair die has

E [X] = 3.5

, which is never rolled; the mean is a balance point, not an attainable outcome.

Spot the error

Var (2 X) = 2 Var (X)

because variance is linear."

The error is treating variance like expectation; variance uses squared deviations, so the constant is squared:

Var (2 X) = 4 Var (X)

Var (X + X) = Var (X) + Var (X) = 2 Var (X)

The sum rule needs independence, but

X

is perfectly correlated with itself; correctly

X + X = 2 X

gives

4 Var (X)

E [1/ X] = 1/ E [X]

since expectation passes through functions."

Only linear functions pass through;

1/ x

is nonlinear, so in general

E [1/ X] \neq = 1/ E [X]

(Jensen even fixes the direction of the gap).

SD (X + Y) = SD (X) + SD (Y)

for independent

X, Y

Variances add, not SDs:

Var (X + Y) = Var (X) + Var (Y)

, so SDs add in quadrature,

SD = σ_{X}^{2} + σ_{Y}^{2}

, which is smaller than

σ_{X} + σ_{Y}

(see Figure 2).

Var (X - Y) = Var (X) - Var (Y)

Subtraction inside doesn't subtract variances;

Var (X - Y) = Var (X) + Var (Y) - 2 Cov (X, Y)

, and for independent variables it adds to

Var (X) + Var (Y)

"For a Bernoulli variable,

E [X^{2}] = (E [X])^{2} = p^{2}

For a

0/1

variable

X^{2} = X

, so

E [X^{2}] = p

, not

p^{2}

; the variance

p - p^{2} = p (1 - p)

is exactly this gap. See Bernoulli and Binomial Distributions.

"Adding a constant

b

shifts the mean and also increases the variance."

Adding

b

slides the whole distribution but leaves spread untouched, so

Var (X + b) = Var (X)

— only the mean

μ

moves.

Cov (X, X) = 1

because a variable is perfectly correlated with itself."

Perfect correlation is

1

, but covariance of

X

with itself is

Var (X)

, which can be any nonnegative number — you're confusing covariance with correlation.

Why questions

Why do we square the deviations instead of just averaging

X - μ

(recall

μ = E [X]

Because

E [X - μ] = 0

always (positive and negative deviations cancel by definition of the mean), so squaring is needed to stop the cancellation and get a real measure of spread — exactly the fold shown in Figure 1.

Why square rather than take absolute value

∣ X - μ ∣

Squaring gives clean, differentiable algebra (a computational formula, additivity under independence) that

∣ \cdot ∣

lacks, and it penalises large deviations more heavily.

Why does the

+ b

vanish in

Var (a X + b)

Because the new mean also shifts by

b

, so in

(a X + b) - (a μ + b)

the two

+ b

terms cancel, leaving only the scaling by

a

Why does independence make covariance zero?

Independence forces

E [X Y] = E [X] E [Y]

, and covariance is defined as exactly

E [X Y] - E [X] E [Y]

, so it collapses to

0

Why is

E [X^{2}]

never smaller than

(E [X])^{2}

Their difference is the variance, an average of squares, which cannot be negative; equality holds only when

X

is constant (the convex-cup gap in Figure 3).

Why does averaging many independent copies shrink the variance?

For the sample mean

\overset{ˉ}{X} = \frac{1}{n} (X_{1} + \dots + X_{n})

, variances add for the sum but the factor

1/ n

squares in the average, giving

Var (\overset{ˉ}{X}) = σ^{2} / n \to 0

— the mechanism behind the Law of Large Numbers.

Why is Bernoulli variance largest at

p = 0.5

p (1 - p)

is a downward parabola peaking at

p = 0.5

; a fair coin is maximally uncertain, while

p

near

0

1

is nearly deterministic and so barely spreads.

Edge cases

What is

Var (X)

when

X

takes a single value with probability

1

Zero — there is no deviation from the mean

μ

, so spread is exactly

0

(the degenerate distribution).

What is

Var (0 \cdot X)

Zero, since

0 \cdot X

is the constant

0

; the scale rule gives

0^{2} Var (X) = 0

, matching a distribution with no spread.

X

and

Y

are uncorrelated but dependent, does

Var (X + Y) = Var (X) + Var (Y)

still hold?

Yes — the sum rule only requires

Cov (X, Y) = 0

, which uncorrelatedness supplies, even though independence (a stronger property) may fail.

Can a distribution have a finite mean but infinite (undefined) variance?

Yes — heavy-tailed distributions can have a well-defined

E [X]

while

E [X^{2}]

diverges, making variance infinite; such cases break the Central Limit Theorem's usual guarantees.

What happens to

SD (a X + b)

a \to 0

It tends to

0

: the distribution is squashed toward the constant

b

, so all spread disappears — consistent with

∣ a ∣ σ \to 0

, where

σ = SD (X)

If two variables are perfectly negatively correlated with equal standard deviation

σ

, what is

Var (X + Y)

Zero — with equal variance

σ^{2}

and

Cov (X, Y) = - σ^{2}

, the sum rule gives

σ^{2} + σ^{2} - 2 σ^{2} = 0

; the fluctuations cancel exactly, like a perfect hedge.