Worked examples — Gram-Schmidt orthogonalization — algorithm

You have seen the machine in the parent note. Here we stress-test it against every kind of input it can ever meet — messy angles, degenerate (dependent) vectors, weird inner products, a real word problem, and an exam twist. Each example tells you which "cell" of the scenario table it kills.

Before we start, one reminder in plain words so nothing is assumed.

---

The scenario matrix

Gram–Schmidt looks like one formula, but the behaviour changes completely depending on what you feed it. Here is every case class that can occur — we will hit every single row below.

#	Cell / scenario	What makes it special	Example
C1	Two clean 2-D vectors, positive overlap	shadow points along $u_1$	Ex 1
C2	Negative overlap ($⟨v_2,u_1⟩<0$)	shadow points opposite $u_1$; sign of $c$ flips	Ex 2
C3	Already orthogonal input	overlap $=0$, algorithm does nothing	Ex 3
C4	Full 3-D, two subtractions	must clear both earlier $u_j$	Ex 4
C5	Degenerate — dependent vector	residual $u_k=0$, cannot normalize	Ex 5
C6	Order matters — same set, swapped $v_1,v_2$	different $u$'s, same subspace	Ex 6
C7	Non-standard inner product (weighted)	$⟨\cdot ,\cdot ⟩$ changes what "perpendicular" means	Ex 7
C8	Word problem — decorrelating measurements	Gram–Schmidt as "remove the common trend"	Ex 8
C9	Exam twist — read off the $R$ of QR decomposition	coefficients ARE the answer	Ex 9

---

Ex 1 — Cell C1: positive overlap in ${\mathbb{R}}^2$

Forecast (guess first!): $v_2$ leans a little along $v_1$ (both point up-right), so the leftover $u_2$ should point up-and-slightly-left. Try to picture it before reading on.

1. $u_1=v_1=(4,3)$. Why this step? The first vector has nothing earlier to be perpendicular to, so we keep it untouched — it defines our first clean axis.

2. Compute the overlap. $⟨v_2,u_1⟩=1\cdot 4+2\cdot 3=10$, and $⟨u_1,u_1⟩=16+9=25$. Why this step? These two numbers form the scalar $c=\frac{10}{25}=0.4$ — the "how many copies of $u_1$ fit inside the shadow of $v_2$".

3. Subtract the shadow. $u_2=(1,2)-0.4(4,3)=(1-1.6,2-1.2)=(-0.6,0.8).$ Why this step? $c=0.4>0$ means the shadow (red arrow in the figure) points the same way as $u_1$; removing it drags $v_2$ back to the perpendicular leftover (pink arrow).

4. Normalize. $\Vert u_1\Vert =\sqrt{25}=5$, $\Vert u_2\Vert =\sqrt{0.36+0.64}=\sqrt{1}=1$ (already unit!). $e_1=\frac{1}{5}(4,3)=(0.8,0.6),e_2=(-0.6,0.8).$

Recall Verify

$⟨u_1,u_2⟩=4(-0.6)+3(0.8)=-2.4+2.4=0$ ✅ perpendicular. $⟨e_1,e_2⟩=0$ and $\Vert e_1\Vert =\Vert e_2\Vert =1$ ✅ orthonormal.

---

Ex 2 — Cell C2: negative overlap (opposite lean)

Forecast: $v_2$ points up and to the left, so it leans against $u_1=(1,0)$. Expect the scalar $c$ to come out negative — the shadow points backward.

1. $u_1=(1,0)$. Why? First vector, kept as-is; it is the positive-$x$ axis.

2. Overlap: $⟨v_2,u_1⟩=(-3)(1)+2\cdot 0=-3$; $⟨u_1,u_1⟩=1$. So $c=-3$. Why this step? A negative $c$ is not an error — it just means the shadow $c{u}_1=(-3,0)$ points in the $-x$ direction, exactly matching $v_2$'s leftward lean.

3. Subtract: $u_2=(-3,2)-(-3)(1,0)=(-3+3,2)=(0,2)$. Why? Removing a backward shadow (subtracting a negative) pushes the residual up onto the $y$-axis — visibly perpendicular to the $x$-axis.

4. Normalize: $\Vert u_2\Vert =2$, so $e_1=(1,0)$, $e_2=(0,1)$.

Recall Verify

$⟨u_1,u_2⟩=1\cdot 0+0\cdot 2=0$ ✅. The output is literally the standard basis — makes sense, any 2 independent 2-D vectors comb down to a rotated pair of axes, and here they align with $x,y$.

---

Ex 3 — Cell C3: input already orthogonal

Forecast: they are already at right angles, so Gram–Schmidt should change nothing but the lengths.

1. $u_1=(2,0)$.
2. Overlap: $⟨v_2,u_1⟩=0\cdot 2+5\cdot 0=0$, so $c=0$. Why this step matters: when the overlap is zero, the shadow is the zero vector — there is nothing to subtract. This is the degenerate-in-a-good-way case: the comb passes straight through.
3. $u_2=(0,5)-0\cdot (2,0)=(0,5)$. Unchanged.
4. Normalize: $e_1=(1,0)$, $e_2=(0,1)$.

Recall Verify

$⟨u_1,u_2⟩=0$ already before we touched it ✅. Gram–Schmidt is idempotent on orthogonal input — a good sanity check that the algorithm never "over-combs".

---

Ex 4 — Cell C4: full ${\mathbb{R}}^3$, two subtractions

Forecast: $u_3$ must be cleaned of two shadows — one on $u_1$, one on $u_2$. Miss either and the perpendicularity proof collapses.

1. $u_1=(1,1,1)$. Why? Base direction.

2. $u_2$: $⟨v_2,u_1⟩=1+1+0=2$, $⟨u_1,u_1⟩=3$. Coefficient $\frac{2}{3}$. $u_2=(1,1,0)-\frac{2}{3}(1,1,1)=\left(\frac{1}{3},\frac{1}{3},-\frac{2}{3}\right).$ Why this step? One earlier clean vector ⇒ one shadow removed.

3. $u_3$ needs BOTH removals. Compute:

   • $⟨v_3,u_1⟩=1+0+0=1$, $⟨u_1,u_1⟩=3$ ⇒ coeff $\frac{1}{3}$.
   • $⟨v_3,u_2⟩=\frac{1}{3}+0+0=\frac{1}{3}$, $⟨u_2,u_2⟩=\frac{1}{9}+\frac{1}{9}+\frac{4}{9}=\frac{6}{9}=\frac{2}{3}$ ⇒ coeff $\frac{1/3}{2/3}=\frac{1}{2}$. $u_3=(1,0,0)-\frac{1}{3}(1,1,1)-\frac{1}{2}\left(\frac{1}{3},\frac{1}{3},-\frac{2}{3}\right).$ Why this step? There are two already-orthogonal vectors, so two shadows must go — the mnemonic "Subtract the Shadows of the Settled."

4. Simplify coordinate-by-coordinate: $u_3=\left(1-\frac{1}{3}-\frac{1}{6},0-\frac{1}{3}-\frac{1}{6},0-\frac{1}{3}+\frac{1}{3}\right)=\left(\frac{1}{2},-\frac{1}{2},0\right).$

Recall Verify

$⟨u_3,u_1⟩=\frac{1}{2}-\frac{1}{2}+0=0$ ✅ $⟨u_3,u_2⟩=\frac{1}{2}\cdot \frac{1}{3}+(-\frac{1}{2})\cdot \frac{1}{3}+0=\frac{1}{6}-\frac{1}{6}=0$ ✅ Both zero ⇒ $u_3$ perpendicular to all earlier vectors, as the induction promised.

---

Ex 5 — Cell C5: degenerate (dependent) input → zero vector

Forecast: $v_2$ lies exactly along $v_1$, so after removing its shadow nothing is left — the residual should be $0$, and normalization will demand dividing by zero.

1. $u_1=(1,2)$.
2. Overlap: $⟨v_2,u_1⟩=2+8=10$, $⟨u_1,u_1⟩=1+4=5$, so $c=2$.
3. Subtract: $u_2=(2,4)-2(1,2)=(0,0)=0$. Why this step? The entire vector was the shadow. Nothing sticks out perpendicular.

Recall Verify

$u_2=(0,0)$ has norm $0$, confirming dependence. Any dependent input produces a zero residual at the offending step.

---

Ex 6 — Cell C6: order matters (swap the inputs)

Forecast: we should get different $u$'s than Ex 1, yet they must span the same plane (${\mathbb{R}}^2$ here) and stay orthonormal. Gram–Schmidt depends on the ordering of the inputs.

1. $u_1=(1,2)$. (In Ex 1 this was $u_2$'s neighbourhood — now it leads.)
2. Overlap: $⟨v_2,u_1⟩=4+6=10$, $⟨u_1,u_1⟩=1+4=5$, $c=2$.
3. Subtract: $u_2=(4,3)-2(1,2)=(2,-1)$. Why this step? First vector fixed the axis; the second gets combed against it.
4. Normalize: $\Vert u_1\Vert =\sqrt{5}$, $\Vert u_2\Vert =\sqrt{5}$. $e_1=\frac{1}{\sqrt{5}}(1,2),e_2=\frac{1}{\sqrt{5}}(2,-1).$

Recall Verify

$⟨u_1,u_2⟩=1\cdot 2+2(-1)=0$ ✅ and $\Vert e_1\Vert =\Vert e_2\Vert =1$ ✅.

---

Ex 7 — Cell C7: a weighted inner product

Forecast: "perpendicular" now means weighted balance, not visual right angles. The numbers change, but the exact same formula runs — this is why the parent note wrote $⟨\cdot ,\cdot ⟩$ abstractly, never "dot".

1. $u_1=(1,0)$.
2. Weighted overlap: $⟨v_2,u_1{⟩}_W=2(1)(1)+3(1)(0)=2$; $⟨u_1,u_1{⟩}_W=2(1)(1)+3\cdot 0=2$. So $c=1$. Why this step? We plug the weighted inner product into the exact same $c=\frac{⟨v,u⟩}{⟨u,u⟩}$ — no new machinery needed.
3. Subtract: $u_2=(1,1)-1\cdot (1,0)=(0,1)$.

Recall Verify

Check with the weighted product: $⟨u_1,u_2{⟩}_W=2(1)(0)+3(0)(1)=0$ ✅. Note: under the ordinary dot product $(1,0)\cdot (0,1)=0$ too here — but in general weighted-orthogonal vectors need not look perpendicular on paper. Orthogonality is defined by the inner product you chose.

---

Ex 8 — Cell C8: word problem (decorrelating trends)

Forecast: B clearly contains a chunk of "flat baseline". Subtracting B's shadow on A should leave a pure rising residual that averages to zero.

1. $u_1=a=(2,2,2)$ — the baseline direction. Why? We treat the baseline as the "settled" reference.
2. Overlap: $⟨b,u_1⟩=2+4+6=12$; $⟨u_1,u_1⟩=4+4+4=12$; $c=1$. Why this step? $c=1$ says B contains exactly one full copy of the baseline.
3. Residual: $u_2=(1,2,3)-1\cdot (2,2,2)=(-1,0,1)$. Why? This is B with the baseline removed — the pure day-to-day change.

Recall Verify

$⟨u_2,u_1⟩=-2+0+2=0$ ✅ independent of baseline. Interpretation check: the residual $(-1,0,1)$ sums to zero ⇒ it carries no average level, only the rising trend. Exactly what "remove the common baseline" should mean. (This is the seed idea of Least squares regression: subtract the projection onto the model you already have.)

---

Ex 9 — Cell C9: exam twist — read off $R$ from QR decomposition

Forecast: every $R$ entry is just an inner product $⟨v_k,e_j⟩$ you already computed — no new work, that's the whole trick.

1. $R_{11}=⟨v_1,e_1⟩=(4,3)\cdot (0.8,0.6)=3.2+1.8=5$. (This is $\Vert v_1\Vert$.)
2. $R_{12}=⟨v_2,e_1⟩=(1,2)\cdot (0.8,0.6)=0.8+1.2=2$.
3. $R_{22}=⟨v_2,e_2⟩=(1,2)\cdot (-0.6,0.8)=-0.6+1.6=1$. (This is $\Vert u_2\Vert$ from Ex 1.)
4. $R_{21}=0$ — always, because $v_1$ has no $e_2$ component ($e_2$ came later). $R=\left(\begin{array}{cc}5& 2\\ 0& 1\end{array}\right).$

Recall Verify

Reconstruct: $QR$ first column $=5e_1=5(0.8,0.6)=(4,3)=v_1$ ✅. Second column $=2e_1+1e_2=(1.6,1.2)+(-0.6,0.8)=(1,2)=v_2$ ✅. So $A=QR$ exactly.

---

Connections

• Orthogonal projection — the shadow used in every example
• Inner product spaces — Ex 7 changes the inner product and the machine still runs
• Linear independence — Ex 5 shows dependence surfacing as a zero vector
• Orthonormal basis — the output of Ex 1, 2, 6
• QR decomposition — Ex 9 reads $R$ straight off the projections
• Least squares — Ex 8 is the "remove what you already model" seed
• Gram-Schmidt orthogonalization — algorithm — the parent this deep-dive expands