1.3.8 · D5Probability & Statistics
Question bank — Expectation, variance, and standard deviation
Before we start, earn every symbol we lean on — nothing below is used before it is pictured.
The picture below is the whole page in one frame: the mean is where a see-saw of probability-weights balances, and variance is the average of the little pink squares hung on each deviation.
Two more small words you'll see in the edge cases:
- A point mass (or "spike") is a value that a random variable takes with some fixed probability, drawn as a single vertical arrow whose height is that probability. A distribution built only from a few such arrows is a discrete distribution.
True or false — justify
A fair coin flip's mean of is a value the coin can actually land on
False — mean is a balance point, not an outcome. A coin only shows or ; the "average" is a summary that no single flip can equal.
Variance can be negative if the data is mostly below the mean
False — variance averages squared distances, and a square is never negative, so regardless of which side of the mean values sit on.
If then is a constant
True — zero average squared distance means every value sits exactly on the mean, so never varies; it is a fixed number.
Standard deviation and variance always have the same units
False — variance is in (e.g. ), while pulls it back to the original units (cm). That rescaling is the whole reason exists.
Adding to every data point leaves the standard deviation unchanged
True — a constant shift slides the whole distribution sideways but keeps every point's distance from the new mean identical, so spread is untouched.
Doubling every data point doubles the variance
False — scaling by multiplies variance by , so doubling () quadruples the variance; it only doubles the standard deviation.
is the same thing as
False — these are equal only when is constant. In general , so squaring-then-averaging exceeds averaging-then-squaring.
For any two random variables,
True — linearity of expectation needs no assumptions about independence; it holds even when and are strongly related.
For any two random variables,
False — this needs independence (or zero covariance). In general there is an extra term , the "move-together" term pictured above.
A distribution with larger spread must have a larger mean
False — mean (location) and variance (spread) are independent knobs. Two distributions can share a mean of yet have wildly different variances.
Spot the error
", because variance is average distance from the mean"
The error is the missing square: variance is , using the squared bars in the opening figure, not the plain distance . The absolute-value version is Mean Absolute Deviation, a different (non-differentiable, outlier-tolerant) measure.
" for independent "
Standard deviations don't add; variances do. Correct: , so and give , not .
""
Wrong power — squaring inside the definition pulls the constant out as : .
"Since always, expectation splits over products"
This factorization holds only for independent . For dependent variables the product's expectation carries their covariance and differs.
""
Standard deviation is never negative; the rule uses absolute value: , so the answer is .
" only when is a probability"
The value doesn't matter — the expectation of any constant is that same constant, since a constant "occurs" with probability .
Why questions
Why do we square deviations instead of just averaging them?
Raw deviations always average to exactly (positives cancel negatives), so they carry no spread information; squaring makes every deviation positive so they accumulate.
Why square rather than take absolute values?
Squaring is smooth/differentiable everywhere and yields clean algebra (linearity, the shortcut). It also weights large deviations more heavily, which is often desirable.
Why does standard deviation exist if we already have variance?
To restore the original units, making the number interpretable — "typically away from the mean" is meaningful, whereas " squared-units away" is not.
Why is expectation called the "center of mass"?
If you placed a weight equal to each value's probability at its position on a ruler, the ruler would balance exactly at — probabilities play the role of masses (exactly the see-saw in the first figure).
Why must relative frequency approach probability for the mean formula to work?
The definition is the limit of the sample average; the law of large numbers guarantees , connecting the two.
Why does feature standardization divide by and not ?
Dividing by (same units as ) makes dimensionless with spread ; dividing by variance would leave awkward reciprocal-units. See 4.1.02-Feature-scaling-and-normalization.
Edge cases
What is the variance of a single guaranteed outcome (probability )?
Zero — there is no spread when only one value can occur, so and .
Can a random variable have a mean it can never actually take?
Yes — the two-spike distribution in the figure above puts probability on and on , giving mean , yet the variable only ever lands on or and never on .
If takes huge values but with vanishing probability, is automatically large?
Not necessarily — each value is weighted by its probability, so tiny probabilities can keep the mean small (though extreme tails can also make diverge; that's a separate pathology).
Does a symmetric distribution guarantee the mean sits at the axis of symmetry?
Yes, provided the mean exists — symmetry makes deviations on both sides cancel, so the balance point coincides with the center of symmetry.
If two independent variables each have variance , what is the variance of their difference ?
Still — variance is shift-and-sign blind to the subtraction: .
As sample size , what happens to the variance of the sample mean of i.i.d. data?
It shrinks like — averaging many independent draws concentrates the estimate, the seed idea behind the 1.3.12-Central-limit-theorem.
Recall One-line self-test
Cover every answer above and re-derive the reasoning, not just the verdict — a correct "false" with a wrong reason still counts as a miss.