Gaussian - Normal distribution properties: Question bank

This bank drills the properties from the parent note: symmetry, moments, affine & additive closure, MLE, and the empirical rule.

True or false — justify

A bigger

σ

makes the peak of the bell curve taller.

False — a bigger

σ

spreads mass out, so the peak

\frac{1}{σ 2 π}

actually gets lower (total area stays 1).

Two different Gaussians can cross each other at exactly one point.

False — setting two Gaussian PDFs equal gives a quadratic (or linear) equation, so they cross at up to two points; equal-variance curves cross once, but "exactly one always" is wrong in general.

X \sim N (μ, σ^{2})

then

- X \sim N (- μ, σ^{2})

True — this is affine closure with

a = - 1, b = 0

: mean becomes

- μ

, variance becomes

(- 1)^{2} σ^{2} = σ^{2}

The sum of two independent Gaussians is Gaussian, and so is their difference.

True — difference

X - Y

is affine (

Y \to - Y

) then additive, giving

N (μ_{1} - μ_{2}, σ_{1}^{2} + σ_{2}^{2})

— variances still add, not subtract.

Skewness and excess kurtosis of a Gaussian are both zero.

True — symmetry kills skewness, and "zero excess kurtosis" is definitional: the Gaussian is the reference against which other tails are compared.

N (μ, σ^{2})

assigns zero probability to any single exact value like

X = μ

True — it's a continuous density, so

P (X = c) = 0

for every point; only intervals carry positive probability.

The 68-95-99.7 rule depends on the specific values of

μ

and

σ

False — after standardizing

z = (x - μ) / σ

the

μ, σ

cancel, so the percentages hold for every Gaussian.

If a sample looks bell-shaped, the data must be exactly Gaussian.

False — bell shape is necessary-looking but not sufficient; tails and kurtosis matter (this is exactly what Q-Q Plots check).

The MLE variance estimator

\overset{σ}{^}_{MLE}^{2} = \frac{1}{n} \sum (x_{i} - \overset{μ}{^})^{2}

is unbiased.

False — it divides by

n

and is biased downward; the unbiased version uses

n - 1

(Bessel's correction).

Uncorrelated jointly-Gaussian variables are independent.

True for the jointly Gaussian case — but uncorrelated marginally-Gaussian variables need not be independent (see Multivariate Gaussian).

Spot the error

"Variance of

X + Y

is always

σ_{1}^{2} + σ_{2}^{2}

Only when

X, Y

are independent (or uncorrelated); in general

Var (X + Y) = σ_{1}^{2} + σ_{2}^{2} + 2 Cov (X, Y)

"To standardize, I subtract the mean and subtract the standard deviation:

Z = (X - μ) - σ

Wrong operation — you divide by

σ

Z = (X - μ) / σ

, which rescales spread to 1; subtracting

σ

doesn't change the spread at all.

Y = 2 X

has variance

2 σ^{2}

The scale factor squares:

Var (2 X) = 2^{2} σ^{2} = 4 σ^{2}

; only the mean scales linearly.

"Averaging

n

measurements makes the variance

n

times smaller, and the standard deviation

n

times smaller."

Variance shrinks by

n

(so

σ^{2} / n

), but standard deviation shrinks only by

n

since std is the square root of variance.

"Maximizing likelihood

L

and maximizing

ln L

can give different

\overset{μ}{^}

No —

ln

is strictly increasing (monotonic), so both peak at the same parameter values; log just turns products into sums.

"Because area under the tail beyond

3 σ

is only

0.3%

, nothing ever lands there."

0.3%

is small but nonzero; over millions of samples such events happen routinely — the tails never truly vanish.

"The maximum-entropy derivation proves nature is Gaussian."

It proves the Gaussian is the least-committed distribution given only a fixed mean and variance; whether nature supplies those constraints is a separate physical question (often via the Central Limit Theorem).

Why questions

Why does the exponent use

(x - μ)^{2}

rather than

∣ x - μ ∣

The squared distance produces smooth symmetric bell-shape and falls out of maximizing entropy under a variance (a squared-deviation) constraint;

∣ x - μ ∣

would give the sharper-peaked Laplace distribution.

Why is

μ

simultaneously the mean, median, and mode of a Gaussian?

Perfect symmetry about

μ

makes half the mass on each side (median) and puts the density peak there (mode), while symmetry also forces the balance point (mean) to coincide.

Why does completing the square appear in the MGF derivation?

It rewrites the messy exponent as "a clean Gaussian bump in

x

" plus "constants in

t

"; the bump integrates to 1, leaving only

e^{μ t + σ^{2} t^{2} /2}

Why do we take the log-likelihood in Maximum Likelihood Estimation?

The joint density is a product of

n

factors;

ln

converts it to a sum, whose derivative is far easier, and monotonicity guarantees the same maximizer.

Why does the empirical rule reduce to the error function

erf

After standardizing, the integral of

e^{- z^{2} /2}

over

[- k, k]

has no elementary antiderivative, so it is defined to be

erf (k / 2)

— a named quantity for that exact integral.

Why must the two Gaussians be independent for the clean additive rule?

Independence forces the covariance term to zero and, more strongly, lets the MGFs multiply —

M_{X + Y} = M_{X} M_{Y}

— which is what produces another Gaussian.

Why does standardizing every Gaussian to

N (0, 1)

make lookup tables possible?

Affine closure means all Gaussians are stretched/shifted copies of the standard one, so a single table of

Φ (z)

answers probability questions for any

μ, σ

Edge cases

What happens to the Gaussian PDF as

σ \to 0

It collapses toward an infinitely tall, infinitely thin spike at

μ

— a point mass (Dirac delta) — the degenerate limit where all probability concentrates at one value.

What happens as

σ \to \infty

The curve flattens toward zero height everywhere, spreading probability so thin it approaches an (improper) uniform "no information" state.

N (μ, 0)

a valid distribution?

Not a proper continuous density — variance zero means the random variable is the constant

μ

with certainty; treat it as a degenerate limit, not a bell curve.

Can

Y = a X + b

stay Gaussian when

a = 0

Then

Y = b

is constant — the degenerate

N (b, 0)

; the affine rule still gives variance

0^{2} σ^{2} = 0

, consistently signalling collapse.

Does MLE make sense with a single data point

n = 1

\overset{μ}{^} = x_{1}

works, but

\overset{σ}{^}_{MLE}^{2} = 0

(all deviation from the one point is zero) — a nonsensical spread estimate, showing you cannot estimate variance from one sample.

If two independent Gaussians have

σ_{1}^{2} = σ_{2}^{2}

but opposite means, does

X - Y

have mean zero and zero variance?

Mean is

μ_{1} - μ_{2}

(nonzero here), and variance is

σ_{1}^{2} + σ_{2}^{2}

(doubled, never cancelled) — differencing never shrinks spread.

What is the probability exactly at the inflection points

μ \pm σ

Zero (single points in a continuous distribution); those points matter as curvature landmarks — where the bell switches from concave-down to concave-up — not as probability masses.

Recall Quick self-test

Bigger $σ$ → taller or shorter peak? ::: Shorter — area is fixed at 1, so spreading out lowers the peak. Variance of $a X + b$ ? ::: $a^{2} σ^{2}$ — the shift $b$ never affects spread. Why divide by $n - 1$ instead of $n$ ? ::: To remove the downward bias of the MLE variance (Bessel's correction).