Worked examples — Exploratory data analysis (EDA) workflow

---

First, earn every symbol

Before a single example, let us make sure nothing is assumed. The parent note wrote this formula:

$r=\frac{{\sum }_{i=1}^n(x_i-\bar{x})(y_i-\bar{y})}{\sqrt{{\sum }_{i=1}^n(x_i-\bar{x}{)}^2}\sqrt{{\sum }_{i=1}^n(y_i-\bar{y}{)}^2}}$

Now let us slowly read the formula out loud, symbol by symbol.

Covariance — the honest name for the top of the fraction

$\sigma$ and $r$ use the SAME $\frac{1}{n}$ — so it cancels

Now watch the $\frac{1}{n}$ vanish. Rewrite $r$ using covariance and the two $\sigma$'s:

$r=\frac{\text{Cov}(X,Y)}{{\sigma }_x{\sigma }_y}=\frac{\frac{1}{n}\sum (x_i-\bar{x})(y_i-\bar{y})}{\sqrt{\frac{1}{n}\sum (x_i-\bar{x}{)}^2}\sqrt{\frac{1}{n}\sum (y_i-\bar{y}{)}^2}}.$

Why $|r|$ can never exceed 1 — the geometric reason

The "unitless shape" story is nice, but why is $r$ trapped in $[-1,+1]$? Here is the visual argument.

This is precisely the Cauchy–Schwarz inequality, $|\vec{u}\cdot \vec{v}|\le \Vert \vec{u}\Vert \Vert \vec{v}\Vert$, which says the dot product can never beat the product of the lengths — geometrically, a projection is never longer than the thing you project. Figure s01 shows the two arrows and the angle.

Read figure s01 (alt-text: two vectors and their angle). On the left panel, the raw data: five blue dots, a red dashed mean-cross at $(\bar{x},\bar{y})$ splitting the plane into four quadrants, and a small green arrow from the cross to each dot — every arrow lands in the "agree" quadrants (both-above or both-below), the visual meaning of $r$ near $+1$. On the right panel, the same data re-drawn as the two big arrows $\vec{u}$ (x-deviations, blue) and $\vec{v}$ (y-deviations, yellow) with the angle $\theta$ between them marked in red; because $\theta$ is small, $\cos \theta =r\approx 0.99$. The narrow angle is the whole reason $r$ is nearly 1.

Everything below uses only these ideas. Prerequisites live in 2.1.15-Statistical-measuresfor-data-analysis (mean, std, median) and 2.1.16-Data-visualization-techniques (scatterplots, boxplots).

---

The scenario matrix

Every correlation / univariate situation falls into one of these cells. The examples that follow are labelled with the cell they cover, and together they hit every row.

#	Case class	What's special	Example
A	Strong positive ($r\to +1$)	points hug a rising line	Ex 1
B	Strong negative ($r\to -1$)	points hug a falling line	Ex 2
C	No linear relation ($r\approx 0$)	shapeless blob	Ex 3
D	Curved but $r\approx 0$ (degenerate meaning)	perfect pattern, zero $r$	Ex 4
E	Zero-spread column ($\sigma =0$) & tiny-$n$	denominator $=0$, $r$ undefined; $n=2$ edge case	Ex 5
F	Outlier hijack	one point flips the number	Ex 6
G	Skew: mean vs median (univariate)	which way the tail points	Ex 7
H	Categorical vs numerical (grouped stats)	word problem / business decision	Ex 8
I	Exam twist: correlation ≠ causation & multicollinearity	interpretation trap	Ex 9
J	Moderate $r$ + sample size / significance	is $r\approx 0.5$ real or luck?	Ex 10

---

Example 1 — Cell A: strong positive correlation

---

Example 2 — Cell B: strong negative correlation

Read figure s02 (alt-text: downward best-fit line hugged by points). The red line is the best-fitting straight line through the five blue dots; it slopes downward. Every dot hugs that line closely — the visual signature of $|r|$ near 1. Compare it mentally with figure s01: same tightness, opposite tilt. Tilt direction = sign of $r$; tightness of the hug = magnitude of $r$.

---

Example 3 — Cell C: no linear relationship

---

Example 4 — Cell D: perfect pattern, but $r=0$ (a trap!)

Read figure s03 (alt-text: dots on a symmetric parabola with mean line). The green curve is the exact parabola $y=x^2$; the blue dots sit perfectly on it — a flawless relationship. Yet the red dashed horizontal line marks $\bar{y}$, and you can see the dots are arranged symmetrically left-and-right about the vertical mean line: for every dot that pulls the product positive on the right, a mirror dot cancels it on the left. That left-right symmetry is why the votes sum to zero.

---

Example 5 — Cell E: a constant column ($\sigma =0$) and the $n=2$ edge case

---

Example 6 — Cell F: one outlier hijacks the number

Read figure s04 (alt-text: four collinear dots plus one far-off red dot). Four blue dots line up on a perfect diagonal — that cluster alone would give $r=1$. The red dot at the bottom-right is the typo $(5,-50)$; the red arrow labels it "one bad row flips r to −0.67". Because that one point sits so far below everything, it single-handedly drags the mean of $y$ down and reverses the sum of products. The lesson is visual: one point far from the crowd can outvote all the others.

---

Example 7 — Cell G: mean vs median tells you the skew direction

---

Example 8 — Cell H: categorical vs numerical (business word problem)

---

Example 9 — Cell I: exam twist — high $r$ that lies

---

Example 10 — Cell J: a moderate $r$ — real signal or lucky noise?

---

Recall

Recall In one sentence, what is

$r$? A single number in $[-1,+1]$ measuring how tightly two columns rise-and-fall together in a straight-line way: $+1$ rising line, $-1$ falling line, $0$ no linear link. Geometrically it is $\cos \theta$ between the two deviation-arrows.

Recall Where does the

$\frac{1}{n}$ go — population vs sample? Both top ($\text{Cov}=\frac{1}{n}\sum \dots$) and bottom (${\sigma }_x{\sigma }_y$, each carrying $\sqrt{\frac{1}{n}}$) contain one factor of $\frac{1}{n}$; they cancel. So $r$ is unaffected by whether you divide by $n$ or not — the constant is irrelevant as long as top and bottom match.

Recall Why can

$|r|$ never exceed 1? Because $r=\cos \theta =\frac{\vec{u}\cdot \vec{v}}{\Vert \vec{u}\Vert \Vert \vec{v}\Vert }$ and Cauchy–Schwarz says $|\vec{u}\cdot \vec{v}|\le \Vert \vec{u}\Vert \Vert \vec{v}\Vert$ — a projection is never longer than the vector itself.

Recall What is covariance and how does it relate to

$r$? $\text{Cov}(X,Y)=\frac{1}{n}\sum (x_i-\bar{x})(y_i-\bar{y})$ — average co-movement in raw units. Then $r=\text{Cov}(X,Y)/({\sigma }_x{\sigma }_y)$: covariance divided by the two spreads to make it unit-free.

Recall

$r=0$ for $y=x^2$. Does that mean no relationship? (Ex 4) No — it means no linear relationship. Pearson $r$ is blind to curves; always plot the scatter.

Recall What happens when a column is constant, and why isn't that

$r=0$? (Ex 5) Its std is $0$, so $\vec{v}$ has zero length and there is no angle to take a cosine of → $r$ is undefined (NaN), not $0$. "$0$" would falsely claim "measured, no link"; the truth is "could not measure". Drop zero-variance columns.

Recall Why is

$|r|$ always $1$ when $n=2$? (Ex 5) Two points determine one line perfectly — no scatter to disagree, so the deviation-arrows are exactly parallel/anti-parallel. Never trust a