Visual walkthrough — Covariance and correlation

We work with two numbers measured on the same object — say height $X$ and weight $Y$ of a person. Each person is one dot on a graph: horizontal position $=X$, vertical position $=Y$.

---

Step 1 — Plot the cloud of dots

WHAT. We collect many $(X,Y)$ pairs and drop each as a dot. This picture is a scatter plot.

WHY. Before any formula, we need to see the raw question: "when $X$ is large, does $Y$ tend to be large too?" A cloud that slopes up-right says yes; a cloud sloping down says opposite; a shapeless blob says no pattern. Everything below is just a way to turn this visual slope into one number.

PICTURE. Look at the up-right tilt of the blue cloud below.

---

Step 2 — Find each variable's balance point $\mu$

WHAT. Compute ${\mu }_X=E[X]$ and ${\mu }_Y=E[Y]$ — the average of all horizontal positions and the average of all vertical positions. Draw the two lines $X={\mu }_X$ (vertical) and $Y={\mu }_Y$ (horizontal).

WHY. "High" and "low" are meaningless until we pick a reference. The mean is the fairest reference: it is the point about which the dots balance. These two lines chop the plane into four rooms (quadrants) that we will label with signs next.

PICTURE. The orange crosshair is the pair $({\mu }_X,{\mu }_Y)$ — the centre of mass of the cloud.

$\underset{\text{avg }X\text{, the vertical line}}{\underbrace{{\mu }_X}}=E[X],\underset{\text{avg }Y\text{, the horizontal line}}{\underbrace{{\mu }_Y}}=E[Y]$

---

Step 3 — Measure each dot as a deviation from the centre

WHAT. For a dot at $(X,Y)$, form its two deviations: $\underset{\text{how far right }(+)\text{ or left }(-)}{\underbrace{X-{\mu }_X}},\underset{\text{how far up }(+)\text{ or down }(-)}{\underbrace{Y-{\mu }_Y}}.$

WHY. Absolute positions don't tell us "agreement". Relative to the centre does: a dot to the right of centre has $X-{\mu }_X>0$; to the left, $<0$. Same idea vertically. This is exactly the raw material of Variance and Standard Deviation — there we squared one deviation; here we will pair two.

PICTURE. The green arrow is the horizontal deviation, the red arrow the vertical deviation, measured from the orange centre to the dot.

---

Step 4 — Multiply the two deviations: a sign for each room

WHAT. For each dot compute the product $(X-{\mu }_X)(Y-{\mu }_Y)$.

WHY. We want one operation that says + for agreement, − for disagreement. Multiplication is the cheapest such operation, because "same sign × same sign = +" and "opposite signs = −". Walk the four rooms:

• Top-right (right and up): $(+)(+)=+$ — they agree (both above average).
• Bottom-left (left and down): $(-)(-)=+$ — they agree (both below average).
• Top-left (left, up): $(-)(+)=-$ — disagreement.
• Bottom-right (right, down): $(+)(-)=-$ — disagreement.

So the product's sign is a vote: $+$ vote for "move together", $-$ vote for "move apart". Its size is how emphatic that dot's vote is (a dot far out in a corner votes loudly).

PICTURE. Each room is shaded by the sign of the product; agreement rooms green $(+)$, disagreement rooms red $(-)$.

---

Step 5 — Average the votes: this IS covariance

WHAT. Take the expectation (average) of all those signed products: $\mathrm{Cov}(X,Y)=E\left[\underset{\text{horiz. deviation}}{\underbrace{(X-{\mu }_X)}}\underset{\text{vert. deviation}}{\underbrace{(Y-{\mu }_Y)}}\right].$

WHY. Averaging lets the green $(+)$ votes and red $(-)$ votes cancel or dominate. Net positive ⇒ the cloud leans up-right (they move together). Net negative ⇒ it leans down-right. Near zero ⇒ no linear tilt. This one number is the whole story of Step 1's picture.

PICTURE. Three clouds side by side: positive covariance (up-tilt), zero (round blob), negative (down-tilt) — with each cloud's net vote shown.

---

Step 6 — The units problem: why covariance can't measure strength

WHAT. Stretch the whole cloud horizontally by a factor $10$ (measure height in millimetres instead of centimetres). The tilt looks the same, but the covariance number multiplies by $10$.

WHY. From bilinearity, $\mathrm{Cov}(aX,Y)=a\mathrm{Cov}(X,Y)$. So a bigger number need not mean a stronger relationship — it might just mean bigger units. We need to divide out each variable's own spread.

PICTURE. Same-shaped cloud, x-axis relabelled — covariance jumps from $0.15$ to $1.5$ with zero change in the pattern.

---

Step 7 — Put covariance on a leash: correlation $\rho$

WHAT. Define ${\rho }_{XY}=\frac{\mathrm{Cov}(X,Y)}{{\sigma }_X{\sigma }_Y}.$

WHY. Dividing by ${\sigma }_X$ rescales the horizontal spread to $1$, and by ${\sigma }_Y$ the vertical spread to $1$. The stretched and unstretched clouds now look identical — and give the same $\rho$. So $\rho$ is a unitless score of pattern shape only, immune to Step 6's trick.

PICTURE. After standardising each axis, the cloud fits inside a box; the tilt of the best-fit line is what $\rho$ reports.

---

Step 8 — Why $\rho$ can never leave $[-1,1]$ (the leash proof, seen)

WHAT. Let $U=X-{\mu }_X$, $V=Y-{\mu }_Y$. For any real slope $t$, the quantity $E[(U-tV{)}^2]$ is an average of squares, so it is $\ge 0$: $\underset{={\sigma }_Y^2}{\underbrace{E[V^2]}}{t}^2-\underset{=2\mathrm{Cov}}{\underbrace{2E[UV]}}t+\underset{={\sigma }_X^2}{\underbrace{E[U^2]}}\ge 0.$

WHY. A quadratic in $t$ that never dips below $0$ can touch the axis at most once — its discriminant must be $\le 0$: $(2\mathrm{Cov}{)}^2-4{\sigma }_Y^2{\sigma }_X^2\le 0⟹{\mathrm{Cov}}^2\le {\sigma }_X^2{\sigma }_Y^2⟹|\rho |\le 1.$ This is exactly the Cauchy–Schwarz Inequality. Equality ($\rho =\pm 1$) happens only when some $t$ makes $E[(U-tV{)}^2]=0$, i.e. $U=tV$ with no scatter — $Y$ is an exactly straight line in $X$.

PICTURE. The parabola $f(t)={\sigma }_Y^2{t}^2-2\mathrm{Cov}t+{\sigma }_X^2$ sitting on or above the axis; when it just kisses the axis, $\rho =\pm 1$.

---

Step 9 — The degenerate cases you must never be ambushed by

WHAT & WHY. Four edge scenarios, each with what $\rho$ does:

• Perfect line, up: all dots on a rising straight line ⇒ $\rho =+1$ (parabola in Step 8 touches axis at $t>0$).
• Perfect line, down: falling straight line ⇒ $\rho =-1$.
• Flat / vertical cloud: if $X$ never varies then ${\sigma }_X=0$, and $\rho =\frac{0}{0}$ is undefined — there is no "tilt" to report.
• Curved but symmetric, e.g. $Y=X^2$ with $X$ symmetric about $0$: the green and red votes cancel exactly, so $\mathrm{Cov}=0$ and $\rho =0$ even though $Y$ is fully determined by $X$. Covariance sees only the linear part of a relationship. This is why $\rho =0$ does not imply Independence of Random Variables.

PICTURE. The four cases in one strip, each labelled with its $\rho$.

---

The one-picture summary

Everything compressed: dots → deviations from the mean → signed products (green agree, red disagree) → average them (covariance) → divide by spreads to get the leashed score $\rho \in [-1,1]$.

Recall Feynman: the whole walkthrough in plain words

Picture two friends on swings, each dot a snapshot in time. First we find each friend's middle position — their balance point. Then for every snapshot we ask two yes/no-ish questions: "is friend $X$ ahead of their middle?" and "is friend $Y$ ahead of theirs?" We multiply the two answers so that both-ahead or both-behind scores $+$, and one-ahead-one-behind scores $-$. Averaging all those little scores is covariance: a big $+$ means great teamwork, a big $-$ means they mirror each other, near $0$ means no pattern. But "how big" the score is depends on how wildly each friend swings, which is unfair for comparing pairs of friends. So we divide by each one's typical swing size — that squeezes the score onto a fixed report card from $-1$ (perfect opposites) through $0$ (no teamwork) to $+1$ (perfect together). That report card is correlation, and a beautiful square-is-never-negative argument (Cauchy–Schwarz) guarantees the score can never escape the $-1$ to $+1$ leash. The only way to hit the very ends is for the dots to lie on a perfectly straight line.

---

Active recall

---

Connections

• Covariance and correlation — the parent topic these pictures unpack.
• Expectation of Random Variables — the averaging machine $E[\cdot ]$ used in every step.
• Variance and Standard Deviation — deviations and $\sigma$ are its building blocks.
• Cauchy–Schwarz Inequality — the leash of Step 8.
• Independence of Random Variables — why $\rho =0$ is weaker than independence.
• Linear Regression — the tilt line of Step 7 made precise.
• Covariance Matrix — all pairwise covariances at once.