Covariance and correlation: Question bank — Maths deep dive

Two shorthands used below (both defined in the parent note):

$Cov (X, Y) = E [(X - μ_{X}) (Y - μ_{Y})] = E [X Y] - E [X] E [Y]$ — the average product of deviations from the mean.
$ρ_{X Y} = \frac{Cov ( X , Y )}{σ _{X} σ _{Y}}$ — that same quantity ==put on a $[- 1, 1]$ leash== by dividing out both standard deviations $σ_{X}, σ_{Y}$ .

True or false — justify

ρ_{X Y} = 0

then

X

and

Y

are independent.

False.

ρ = 0

only kills linear co-movement; a symmetric quadratic link like

Y = X^{2}

(with

X

symmetric about 0) has

ρ = 0

yet

Y

is fully determined by

X

. See Independence of Random Variables.

X, Y

are independent then

Cov (X, Y) = 0

True. Independence gives

E [X Y] = E [X] E [Y]

, so

E [X Y] - E [X] E [Y] = 0

. The arrow runs only this direction, never back.

A covariance of

500

means a stronger relationship than a covariance of

2

False. Covariance carries units and scales with the variables' magnitudes;

500 kg\cdotcm

2 m\cdots

aren't comparable. Only the unitless

ρ

measures strength.

Cov (X, X)

can be negative.

False.

Cov (X, X) = E [X^{2}] - (E [X])^{2} = Var (X) \geq 0

; a variable always "agrees" perfectly with itself.

ρ_{X Y} = 1

means the data points lie exactly on a straight line.

True.

ρ = \pm 1

is the equality case of Cauchy–Schwarz, which forces

Y = a X + b

exactly;

+ 1

means positive slope,

- 1

negative. See Cauchy–Schwarz Inequality.

Multiplying

X

- 3

flips the sign of the correlation but not its magnitude.

True.

ρ_{a X, Y} = sign (a) ρ_{X Y}

; the factor

3

cancels in numerator and denominator, only the sign of

a = - 3

survives.

Var (X + Y) = Var (X) + Var (Y)

always.

False. It equals

Var (X) + Var (Y) + 2 Cov (X, Y)

; the identity only holds when

Cov (X, Y) = 0

(e.g. under independence).

Correlation of

0.9

between ice-cream sales and drownings proves ice cream causes drowning.

False. Correlation is not causation; a lurking common cause (hot summer weather) can drive both variables up together with no direct link.

Adding a constant

b

to every value of

X

changes

Cov (X, Y)

False. Covariance ignores location:

Cov (X + b, Y) = Cov (X, Y)

because the shift

b

cancels inside the deviation

X - μ_{X}

ρ_{X Y} = 0.6

means

60%

of the points show the trend.

False.

ρ

is not a percentage of points; it is a scaled average of deviation products. The "variance explained" quantity is

ρ^{2} = 0.36

, and even that is not a count of points. See Linear Regression.

Spot the error

Cov (2 X + 5, 3 Y - 1) = 2 \cdot 3 \cdot Cov (X, Y) + 5 - 1

Wrong. Shifts contribute nothing: the correct value is

2 \cdot 3 \cdot Cov (X, Y) = 6 Cov (X, Y)

. The constants

+ 5, - 1

vanish by bilinearity.

"Since

ρ \in [- 1, 1]

, covariance also lives in

[- 1, 1]

Wrong. Only

ρ

is bounded (it's the rescaled version). Covariance is unbounded and can be any real number, positive or negative, of any magnitude.

X, Y

have

Cov = 0

, so

Var (X - Y) = Var (X) - Var (Y)

Wrong. Variance never subtracts:

Var (X - Y) = Var (X) + Var (Y) - 2 Cov (X, Y)

, and with

Cov = 0

that's

Var (X) + Var (Y)

— a sum.

ρ = - 1

is weaker than

ρ = + 1

because it's negative."

Wrong. Strength is

∣ ρ ∣

; both

- 1

and

+ 1

are perfectly linear. The sign only says which direction (opposite vs same), not how strong.

"They computed

Cov (X, Y) = E [X Y] - E [X] - E [Y]

Wrong. The subtracted term is the product of the means, not their difference:

Cov = E [X Y] - E [X] E [Y]

ρ^{2} = 1.2

for this dataset."

Wrong. Impossible:

∣ ρ ∣ \leq 1

forces

ρ^{2} \leq 1

. A value above

1

signals an arithmetic error, likely dividing by a wrong

σ

Why questions

Why do we multiply the two deviations instead of, say, adding them?

Multiplication is the cheapest operation that returns

+

when both deviations share a sign (agreement) and

-

when they differ (disagreement); adding would just recover the separate means and lose the co-movement.

Why divide covariance by

σ_{X} σ_{Y}

to get correlation?

To strip out units and scale, so relationships in kg·cm and rupees·seconds become comparable; the division caps the result in

[- 1, 1]

by Cauchy–Schwarz.

Why does the Cauchy–Schwarz proof start from

E [(U - t V)^{2}] \geq 0

A squared quantity is never negative for any

t

, so this quadratic-in-

t

can never dip below zero, forcing its discriminant

\leq 0

— exactly the inequality

Cov^{2} \leq σ_{X}^{2} σ_{Y}^{2}

. See Cauchy–Schwarz Inequality.

Why is covariance called a generalisation of variance?

Setting

Y = X

collapses the definition to

E [(X - μ_{X})^{2}] = Var (X)

; variance is just covariance of a variable with itself. See Variance and Standard Deviation.

Why can independence force

Cov = 0

but

Cov = 0

cannot force independence?

Independence controls the whole joint distribution (

E [X Y] = E [X] E [Y]

and much more), while covariance only inspects the linear/first-moment co-movement — a narrow slice that can be zero while nonlinear dependence survives.

Why does correlation ignore how big the swings are?

Because dividing by each

σ

normalises each variable to its own spread, leaving only how well they match, not how far they move.

Why is

ρ

preferred over

Cov

when comparing two different relationships?

Cov

changes if you merely rescale units (multiply

X

by 1000 and it jumps 1000×), whereas

ρ

is scale-invariant, so it isolates genuine strength.

Edge cases

What is

ρ

when

Y

is a constant (say

Y = 7

always)?

Undefined. A constant has

σ_{Y} = 0

, so we'd divide by zero; also

Cov (X, 7) = 0

since a constant never deviates. There is simply no linear trend to measure.

What is

Cov (X, Y)

if the entire distribution sits at a single point

(a, b)

Zero (in fact degenerate): both deviations are always

0

, and both variances are

0

, so covariance is

0

and

ρ

is undefined.

Y = - 2 X + 7

exactly, what are

Cov (X, Y)

's sign and

ρ

ρ = - 1

(perfect negative linear);

Cov (X, Y) = - 2 Var (X) < 0

, negative because the slope

- 2

is negative and

Var (X) > 0

Can

ρ = + 1

while the two variables live on wildly different scales (heights in mm vs km)?

Yes.

ρ

is scale-invariant, so a perfect line

Y = a X + b

gives

ρ = + 1

regardless of units — only the sign of the slope

a

matters.

For symmetric

X

(say uniform on

{- 1, 0, 1}

) and

Y = ∣ X ∣

, is

Cov = 0

Yes,

Cov = 0

, yet

Y

is fully determined by

X

. Like

Y = X^{2}

, the dependence is symmetric and nonlinear, so it's invisible to covariance.

What does

Var (X + Y)

become in the extreme

ρ = - 1

with

σ_{X} = σ_{Y} = σ

Var (X + Y) = σ^{2} + σ^{2} + 2 (- σ^{2}) = 0

— the two variables cancel perfectly, so their sum is a constant with no spread.

If you scale

X

by a factor

c \to \infty

, what happens to

Cov (X, Y)

versus

ρ_{X Y}

Cov

blows up like

c Cov (X, Y) \to \pm \infty

, while

ρ

stays fixed — the

c

cancels between numerator and

σ_{c X} = ∣ c ∣ σ_{X}

Connections

Covariance and Correlation — the parent note these traps drill.
Variance and Standard Deviation — the degenerate $σ = 0$ cases live here.
Independence of Random Variables — the one-way arrow behind half these traps.
Cauchy–Schwarz Inequality — why $∣ ρ ∣ \leq 1$ and why $1.2$ is impossible.
Linear Regression — $ρ^{2}$ as variance explained, not a point count.
Covariance Matrix — where these pairwise facts scale up to vectors.