Visual walkthrough — Orthogonal sets and orthonormal basis

This page is the picture-by-picture derivation behind Orthogonal sets and orthonormal basis. Where the parent note stated the coordinate formula, here we earn it, one drawing at a time.

---

Step 1 — What is a vector, and what does "dot" measure?

WHAT. A vector is just an arrow from the origin (the point $(0,0)$) to some spot. We write $\mathbf{y}$ in bold to mean "the whole arrow", and its tip's coordinates in a column, e.g. $\mathbf{y}=\left[\begin{array}{c}4\\ 3\end{array}\right]$ means "go 4 right, 3 up."

WHY the dot product. We are hunting for a machine that eats two arrows and spits out a single number telling us how much they point the same way. That machine is the dot product. For two arrows $\mathbf{a}=\left[\begin{array}{c}{a}_1\\ a_2\end{array}\right]$ and $\mathbf{b}=\left[\begin{array}{c}{b}_1\\ b_2\end{array}\right]$:

$\underset{\text{one number out}}{\underbrace{\mathbf{a}\cdot \mathbf{b}}}=\underset{\text{a’s }x}{\underbrace{a_1}}\underset{\text{b’s }x}{\underbrace{b_1}}+\underset{\text{a’s }y}{\underbrace{a_2}}\underset{\text{b’s }y}{\underbrace{b_2}}$

Multiply matching slots, then add. There is a second face of this same number that we will need constantly:

$\mathbf{a}\cdot \mathbf{b}=\underset{\text{length of }a}{\underbrace{\Vert \mathbf{a}\Vert }}\underset{\text{length of }b}{\underbrace{\Vert \mathbf{b}\Vert }}\underset{\text{how aligned}}{\underbrace{\cos \theta }}$

Here $\Vert \mathbf{a}\Vert$ is the length of the arrow (Pythagoras: $\Vert \mathbf{a}\Vert =\sqrt{a_1^2+a_2^2}$), and $\theta$ is the angle between the two arrows. The key fact we will lean on: when the arrows are perpendicular, $\theta =90^{\circ }$, and $\cos 90^{\circ }=0$, so the whole dot product is $0$.

PICTURE. Two arrows and the angle between them; the dot number shrinks toward $0$ as they swing toward a right angle.

---

Step 2 — What an "orthogonal basis" looks like

WHAT. A basis is a set of arrows we use as axes — every point is described by "how far along each axis." An orthogonal basis demands the axes be mutually perpendicular. Call them ${\mathbf{u}}_1,{\mathbf{u}}_2,\dots$ Perpendicular means, by Step 1,

${\mathbf{u}}_i\cdot {\mathbf{u}}_j=0\text{for }i\ne j(\text{different axes}\Rightarrow \text{dot is }0)$

WHY we bother. Ordinary axes can lean at any angle. When they lean, the coordinates tangle together and you must solve equations. When they sit at a clean right angle, each coordinate becomes independent of the others — that is the payoff we are chasing.

PICTURE. Left: a slanted generic basis (axes not perpendicular) — messy. Right: an orthogonal basis (axes at $90^{\circ }$) — clean.

---

Step 3 — The goal: write $\mathbf{y}$ as a recipe of the axes

WHAT. Take any target arrow $\mathbf{y}$ living in the space spanned by our axes. Because they form a basis, some amounts $c_1,c_2$ exist so that

$\mathbf{y}=\underset{\text{amount of }{u}_1}{\underbrace{c_1}}{\mathbf{u}}_1+\underset{\text{amount of }{u}_2}{\underbrace{c_2}}{\mathbf{u}}_2$

WHY this is only half an answer. We know the $c$'s exist, but we do not yet know their values. For a slanted basis, extracting $c_1$ and $c_2$ means solving two coupled equations. Our whole mission: show that perpendicularity lets us pluck out each $c_j$ alone.

PICTURE. $\mathbf{y}$ drawn as the diagonal of the parallelogram built from $c_1{\mathbf{u}}_1$ and $c_2{\mathbf{u}}_2$.

---

Step 4 — The magic move: dot both sides with one axis

WHAT. Take the equation from Step 3 and dot everything with a single chosen axis ${\mathbf{u}}_1$:

$\mathbf{y}\cdot {\mathbf{u}}_1=(c_1{\mathbf{u}}_1+c_2{\mathbf{u}}_2)\cdot {\mathbf{u}}_1=c_1({\mathbf{u}}_1\cdot {\mathbf{u}}_1)+c_2(\underset{=0\text{by Step 2}}{\underbrace{{\mathbf{u}}_2\cdot {\mathbf{u}}_1}})$

WHY this exact tool. The dot product is the only operation that can "ask an axis how much it sees." Because ${\mathbf{u}}_2\perp {\mathbf{u}}_1$, the cross term ${\mathbf{u}}_2\cdot {\mathbf{u}}_1$ is exactly zero — the $c_2$ contribution vanishes. One perpendicularity fact just deleted an unknown. What survives is a single clean equation in $c_1$ alone:

$\mathbf{y}\cdot {\mathbf{u}}_1=c_1({\mathbf{u}}_1\cdot {\mathbf{u}}_1)$

PICTURE. The $c_2{\mathbf{u}}_2$ piece is drawn perpendicular to ${\mathbf{u}}_1$; its "shadow" on the ${\mathbf{u}}_1$ line has length $0$ — it contributes nothing to $\mathbf{y}\cdot {\mathbf{u}}_1$.

---

Step 5 — Solve for the coordinate (it decouples!)

WHAT. Divide the survivor equation by the number ${\mathbf{u}}_1\cdot {\mathbf{u}}_1$:

$c_1=\frac{\stackrel{\text{how much }y\text{ leans along }{u}_1}{\overbrace{\mathbf{y}\cdot {\mathbf{u}}_1}}}{\underset{=\Vert {\mathbf{u}}_1{\Vert }^2,\text{axis’s own size}}{\underbrace{{\mathbf{u}}_1\cdot {\mathbf{u}}_1}}}$

The same argument with ${\mathbf{u}}_2$ gives $c_2=\frac{\mathbf{y}\cdot {\mathbf{u}}_2}{{\mathbf{u}}_2\cdot {\mathbf{u}}_2}$, and in general:

WHY the denominator. ${\mathbf{u}}_j\cdot {\mathbf{u}}_j=\Vert {\mathbf{u}}_j{\Vert }^2$ measures how long axis $j$ is. The numerator $\mathbf{y}\cdot {\mathbf{u}}_j$ mixes in the axis's length too, so we divide it back out — otherwise a longer ruler would report a wrong count. Geometrically this is exactly the Orthogonal projection of $\mathbf{y}$ onto the ${\mathbf{u}}_j$-line.

PICTURE. Drop a perpendicular from $\mathbf{y}$'s tip onto the ${\mathbf{u}}_1$ line; the foot marks $c_1{\mathbf{u}}_1$. The height of the numerator becomes a length along the axis after dividing.

---

Step 6 — The orthonormal shortcut (denominators disappear)

WHAT. If we first shrink every axis to length $1$ (called normalizing: replace ${\mathbf{u}}_j$ by ${\mathbf{q}}_j={\mathbf{u}}_j/\Vert {\mathbf{u}}_j\Vert$), then ${\mathbf{q}}_j\cdot {\mathbf{q}}_j=1$ and the denominator becomes $1$:

$c_j=\mathbf{y}\cdot {\mathbf{q}}_j(\text{denominator }=1)$

WHY. Dividing an arrow by its own length keeps its direction but forces its length to exactly $1$. With unit axes, "how much $\mathbf{y}$ leans along ${\mathbf{q}}_j$" is the coordinate — no rescaling needed.

PICTURE. A length-$1$ axis with $\mathbf{y}$'s shadow landing directly as the coordinate.

---

Step 7 — Edge & degenerate cases (never leave the reader stranded)

WHAT / WHY. Three scenarios can break the recipe — here is what happens in each.

• Zero axis (${\mathbf{u}}_j=0$). Then ${\mathbf{u}}_j\cdot {\mathbf{u}}_j=0$ and we would divide by zero. This is why the parent note insists the vectors be nonzero. The zero arrow points nowhere and cannot be an axis.
• $\mathbf{y}$ perpendicular to an axis. Then $\mathbf{y}\cdot {\mathbf{u}}_j=0$, so $c_j=0$ — perfectly fine, it just means $\mathbf{y}$ has no component along that axis. Not a failure.
• $\mathbf{y}$ outside the span (fewer axes than dimensions). The formula still runs and returns the best approximation of $\mathbf{y}$ inside the span — the piece the axes can see. The leftover perpendicular part is invisible to every dot product. This is the seed of Least squares.

PICTURE. Three mini-panels: (a) a zero axis crossed out, (b) $\mathbf{y}\perp {\mathbf{u}}_1$ giving $c_1=0$, (c) $\mathbf{y}$ sticking out of a line, its shadow (projection) plus a red leftover.

---

The one-picture summary

Everything above compressed: $\mathbf{y}$, two perpendicular axes, the two perpendicular drops giving $c_1$ and $c_2$, each labelled with its dot-product formula. Read it right-to-left as "shadow, divide, done."

Recall Feynman retelling — say it back in plain words

Imagine two rulers laid on the floor at a perfect right angle. I have an arrow pointing to some spot and I want to know how far along each ruler it reaches. Trick: I shine a light straight down onto ruler one — the shadow's length is how much of my arrow lies along ruler one, and the dot product $\mathbf{y}\cdot {\mathbf{u}}_1$ computes that shadow. Ruler two is at $90^{\circ }$, so it casts no shadow on ruler one — that is why the two answers don't interfere and I never solve equations. I do have to divide by the ruler's own length (${\mathbf{u}}_1\cdot {\mathbf{u}}_1$) so a long ruler doesn't over-count; if I pre-cut every ruler to length $1$, even that division vanishes. Zero-length rulers are banned (divide by zero), a shadow of length zero just means my arrow is perpendicular to that ruler, and if my arrow points out of the floor entirely, the shadows give me the closest in-floor arrow — the rest is lost to sight. That last fact is the whole idea of least squares.

Prerequisites & next steps: built on Dot product and norms and Orthogonal projection; the machinery to manufacture an orthogonal basis is Gram-Schmidt process; packaging orthonormal axes into a matrix leads to QR decomposition; the "leftover" idea powers Least squares; symmetric matrices give orthonormal axes for free via Eigenvalues and eigenvectors.