Exercises — Orthogonal sets and orthonormal basis

What we check: every distinct pair's dot product, because orthogonal means "each pair perpendicular."

• ${\mathbf{u}}_1\cdot {\mathbf{u}}_2=(2)(0)+(-1)(1)+(1)(1)=0.$ ✓
• ${\mathbf{u}}_1\cdot {\mathbf{u}}_3=(2)(1)+(-1)(1)+(1)(-1)=2-1-1=0.$ ✓
• ${\mathbf{u}}_2\cdot {\mathbf{u}}_3=(0)(1)+(1)(1)+(1)(-1)=0.$ ✓

All three pairwise dots are $0$ → orthogonal. Orthonormal? Check lengths: $\Vert {\mathbf{u}}_1{\Vert }^2=4+1+1=6\ne 1$. Not unit length, so not orthonormal (would need to divide each by its length).

Why the shortcut: the basis is orthogonal, so each coordinate decouples: $c_j=\frac{\mathbf{y}\cdot {\mathbf{u}}_j}{{\mathbf{u}}_j\cdot {\mathbf{u}}_j}$ (no system to solve). Keep the denominator because the vectors are not unit length.

• $\mathbf{y}\cdot {\mathbf{u}}_1=6+0+6=12$, ${\mathbf{u}}_1\cdot {\mathbf{u}}_1=6\Rightarrow c_1=12/6=2.$
• $\mathbf{y}\cdot {\mathbf{u}}_2=0+0+6=6$, ${\mathbf{u}}_2\cdot {\mathbf{u}}_2=0+1+1=2\Rightarrow c_2=6/2=3.$
• $\mathbf{y}\cdot {\mathbf{u}}_3=3+0-6=-3$, ${\mathbf{u}}_3\cdot {\mathbf{u}}_3=1+1+1=3\Rightarrow c_3=-3/3=-1.$

Check: $2{\mathbf{u}}_1+3{\mathbf{u}}_2-{\mathbf{u}}_3=\left[\begin{array}{c}4+0-1\\ -2+3-1\\ 2+3+1\end{array}\right]=\left[\begin{array}{c}3\\ 0\\ 6\end{array}\right]=\mathbf{y}.$ ✓ Answer: $(c_1,c_2,c_3)=(2,3,-1)$.

Why divide by length: to shrink/grow the arrow to length $1$ without turning it. $\Vert {\mathbf{u}}_2\Vert =\sqrt{0+1+1}=\sqrt{2}$, so ${\mathbf{q}}_2=\frac{1}{\sqrt{2}}\left[\begin{array}{c}0\\ 1\\ 1\end{array}\right]=\left[\begin{array}{c}0\\ 1/\sqrt{2}\\ 1/\sqrt{2}\end{array}\right].$ Check: ${\mathbf{q}}_2\cdot {\mathbf{q}}_2=0+\frac{1}{2}+\frac{1}{2}=1.$ ✓

The deciding question: is the pair orthogonal? The shortcut $c_j=\frac{\mathbf{y}\cdot {\mathbf{u}}_j}{{\mathbf{u}}_j\cdot {\mathbf{u}}_j}$ is derived from orthogonality — it isolates one coefficient only because all cross-terms ${\mathbf{u}}_i\cdot {\mathbf{u}}_j$ ($i\ne j$) vanish.

• ${\mathbf{v}}_1\cdot {\mathbf{v}}_2=1-1=0$ → orthogonal, shortcut valid.
• ${\mathbf{w}}_1\cdot {\mathbf{w}}_2=1+0=1\ne 0$ → not orthogonal, shortcut invalid; you'd have to solve a system.

Coordinates in the $\mathbf{v}$-basis:

• $c_1=\frac{\mathbf{y}\cdot {\mathbf{v}}_1}{{\mathbf{v}}_1\cdot {\mathbf{v}}_1}=\frac{4+2}{2}=3.$
• $c_2=\frac{\mathbf{y}\cdot {\mathbf{v}}_2}{{\mathbf{v}}_2\cdot {\mathbf{v}}_2}=\frac{4-2}{2}=1.$

Check: $3{\mathbf{v}}_1+1{\mathbf{v}}_2=\left[\begin{array}{c}3+1\\ 3-1\end{array}\right]=\left[\begin{array}{c}4\\ 2\end{array}\right]=\mathbf{y}.$ ✓ Answer: $(3,1)$.

Take ${\mathbf{u}}_1=\left[\begin{array}{c}1\\ 0\end{array}\right]$ and ${\mathbf{u}}_2=\left[\begin{array}{c}0\\ 0\end{array}\right]=0$. Orthogonal? ${\mathbf{u}}_1\cdot {\mathbf{u}}_2=0$ ✓ (the zero vector is perpendicular to everything, since every product term is $0$). Independent? No: $0\cdot {\mathbf{u}}_1+5\cdot {\mathbf{u}}_2=0$ is a nontrivial relation (coefficient $5\ne 0$) that still sums to zero — because ${\mathbf{u}}_2$ contributes nothing. So the set is dependent. Where the proof needs "nonzero": the independence argument ends at $c_j\Vert {\mathbf{u}}_j{\Vert }^2=0$. If ${\mathbf{u}}_j=0$ then $\Vert {\mathbf{u}}_j{\Vert }^2=0$, and $c_j\cdot 0=0$ holds for any $c_j$ — it cannot force $c_j=0$. The picture: a zero-length arrow points nowhere, so it carries no independent direction.

(a) Normalize. Lengths: $\Vert {\mathbf{a}}_1\Vert =\sqrt{1+1}=\sqrt{2}$, $\Vert {\mathbf{a}}_2\Vert =\sqrt{2}$, $\Vert {\mathbf{a}}_3\Vert =\sqrt{4}=2$. ${\mathbf{q}}_1=\frac{1}{\sqrt{2}}\left[\begin{array}{c}1\\ 1\\ 0\end{array}\right],{\mathbf{q}}_2=\frac{1}{\sqrt{2}}\left[\begin{array}{c}1\\ -1\\ 0\end{array}\right],{\mathbf{q}}_3=\left[\begin{array}{c}0\\ 0\\ 1\end{array}\right].$ (b) Coordinates — denominators are now $1$, so just dot:

• $c_1=\mathbf{y}\cdot {\mathbf{q}}_1=\frac{1}{\sqrt{2}}(5+1+0)=\frac{6}{\sqrt{2}}=3\sqrt{2}.$
• $c_2=\mathbf{y}\cdot {\mathbf{q}}_2=\frac{1}{\sqrt{2}}(5-1+0)=\frac{4}{\sqrt{2}}=2\sqrt{2}.$
• $c_3=\mathbf{y}\cdot {\mathbf{q}}_3=4.$

Check (reassemble): $3\sqrt{2}{\mathbf{q}}_1=3\sqrt{2}\cdot \frac{1}{\sqrt{2}}(1,1,0)=(3,3,0)$; $2\sqrt{2}{\mathbf{q}}_2=(2,-2,0)$; $4{\mathbf{q}}_3=(0,0,4)$. Sum $=(5,1,4)=\mathbf{y}.$ ✓

$U=\left[\begin{array}{ccc}1/\sqrt{2}& 1/\sqrt{2}& 0\\ 1/\sqrt{2}& -1/\sqrt{2}& 0\\ 0& 0& 1\end{array}\right]$. $U\mathbf{x}=\left[\begin{array}{c}2/\sqrt{2}+0+0\\ 2/\sqrt{2}+0+0\\ 3\end{array}\right]=\left[\begin{array}{c}\sqrt{2}\\ \sqrt{2}\\ 3\end{array}\right]$. $\Vert U\mathbf{x}{\Vert }^2=2+2+9=13$. And $\Vert \mathbf{x}{\Vert }^2=4+0+9=13$. Equal ✓. Why it had to: $\Vert U\mathbf{x}{\Vert }^2=(U\mathbf{x})\cdot (U\mathbf{x})={\mathbf{x}}^{\mathsf{T}}{U}^{\mathsf{T}}U\mathbf{x}={\mathbf{x}}^{\mathsf{T}}I\mathbf{x}=\Vert \mathbf{x}{\Vert }^2$, using $U^{\mathsf{T}}U=I$ (orthonormal columns). Geometrically $U$ is a rotation/reflection — it turns the arrow, never stretches it.

Proof. Expand the length using bilinearity of the dot product: $\Vert \mathbf{y}{\Vert }^2=\mathbf{y}\cdot \mathbf{y}=\left({\sum }_i{c}_i{\mathbf{q}}_i\right)\cdot \left({\sum }_j{c}_j{\mathbf{q}}_j\right)={\sum }_i{\sum }_j{c}_i{c}_j({\mathbf{q}}_i\cdot {\mathbf{q}}_j).$ Why the double sum collapses: orthonormality means ${\mathbf{q}}_i\cdot {\mathbf{q}}_j=0$ unless $i=j$, and $=1$ when $i=j$. Every off-diagonal term dies; each diagonal term keeps $c_i{c}_i\cdot 1$: $\Vert \mathbf{y}{\Vert }^2={\sum }_i{c}_i^2.■$ This is Pythagoras in $p$ dimensions: the coordinates are the legs of a right-angled box, and $\Vert \mathbf{y}\Vert$ is the diagonal. Verify on L4.1: $c_1^2+c_2^2+c_3^2=(3\sqrt{2}{)}^2+(2\sqrt{2}{)}^2+4^2=18+8+16=42$. Directly $\Vert \mathbf{y}{\Vert }^2=5^2+1^2+4^2=25+1+16=42.$ ✓

Why this formula: with an orthonormal basis of $W$, the closest point of $W$ to $\mathbf{y}$ is found by dotting — the same cheap trick, now used to project instead of to give exact coordinates.

• $\mathbf{y}\cdot {\mathbf{q}}_1=\frac{1}{\sqrt{2}}(4+2+0)=\frac{6}{\sqrt{2}}=3\sqrt{2}.$
• $\mathbf{y}\cdot {\mathbf{q}}_2=5.$ $\hat{\mathbf{y}}=3\sqrt{2}\cdot \frac{1}{\sqrt{2}}\left[\begin{array}{c}1\\ 1\\ 0\end{array}\right]+5\left[\begin{array}{c}0\\ 0\\ 1\end{array}\right]=\left[\begin{array}{c}3\\ 3\\ 0\end{array}\right]+\left[\begin{array}{c}0\\ 0\\ 5\end{array}\right]=\left[\begin{array}{c}3\\ 3\\ 5\end{array}\right].$ Error: $\mathbf{y}-\hat{\mathbf{y}}=\left[\begin{array}{c}4-3\\ 2-3\\ 5-5\end{array}\right]=\left[\begin{array}{c}1\\ -1\\ 0\end{array}\right]$. Perpendicular to $W$? Dot with each basis vector:
• $(\mathbf{y}-\hat{\mathbf{y}})\cdot {\mathbf{q}}_1=\frac{1}{\sqrt{2}}(1-1+0)=0.$ ✓
• $(\mathbf{y}-\hat{\mathbf{y}})\cdot {\mathbf{q}}_2=0+0+0=0.$ ✓

The error is orthogonal to both spanning vectors, hence to all of $W$ — so $\hat{\mathbf{y}}$ is genuinely the closest point. This is the seed of Least squares and, stacking the ${\mathbf{q}}_j$ as columns, of QR decomposition.

L1 check all pairs ::: L2 coordinate $=\frac{\mathbf{y}\cdot {\mathbf{u}}_j}{{\mathbf{u}}_j\cdot {\mathbf{u}}_j}$, keep denominator ::: L3 shortcut needs orthogonality & nonzero vectors ::: L4 orthonormal ⇒ drop denominator & lengths preserved ::: L5 $\Vert \mathbf{y}{\Vert }^2=\sum c_j^2$ and projection by dotting.