Exercises — Orthogonal sets and orthonormal basis

Hum kya check karte hain: har alag pair ka dot product, kyunki orthogonal ka matlab hai "har pair perpendicular."

• ${\mathbf{u}}_1\cdot {\mathbf{u}}_2=(2)(0)+(-1)(1)+(1)(1)=0.$ ✓
• ${\mathbf{u}}_1\cdot {\mathbf{u}}_3=(2)(1)+(-1)(1)+(1)(-1)=2-1-1=0.$ ✓
• ${\mathbf{u}}_2\cdot {\mathbf{u}}_3=(0)(1)+(1)(1)+(1)(-1)=0.$ ✓

Teeno pairwise dots $0$ hain → orthogonal. Orthonormal? Lengths check karo: $\Vert {\mathbf{u}}_1{\Vert }^2=4+1+1=6\ne 1$. Unit length nahi hai, isliye orthonormal nahi (har ek ko uski length se divide karna padega).

Shortcut kyun kaam karta hai: basis orthogonal hai, isliye har coordinate decouple ho jaata hai: $c_j=\frac{\mathbf{y}\cdot {\mathbf{u}}_j}{{\mathbf{u}}_j\cdot {\mathbf{u}}_j}$ (koi system solve nahi karna). Denominator rakhna zaroori hai kyunki vectors unit length ke nahi hain.

• $\mathbf{y}\cdot {\mathbf{u}}_1=6+0+6=12$, ${\mathbf{u}}_1\cdot {\mathbf{u}}_1=6\Rightarrow c_1=12/6=2.$
• $\mathbf{y}\cdot {\mathbf{u}}_2=0+0+6=6$, ${\mathbf{u}}_2\cdot {\mathbf{u}}_2=0+1+1=2\Rightarrow c_2=6/2=3.$
• $\mathbf{y}\cdot {\mathbf{u}}_3=3+0-6=-3$, ${\mathbf{u}}_3\cdot {\mathbf{u}}_3=1+1+1=3\Rightarrow c_3=-3/3=-1.$

Check: $2{\mathbf{u}}_1+3{\mathbf{u}}_2-{\mathbf{u}}_3=\left[\begin{array}{c}4+0-1\\ -2+3-1\\ 2+3+1\end{array}\right]=\left[\begin{array}{c}3\\ 0\\ 6\end{array}\right]=\mathbf{y}.$ ✓ Answer: $(c_1,c_2,c_3)=(2,3,-1)$.

Length se divide kyun karte hain: arrow ko length $1$ tak shrink/grow karne ke liye, bina use turn kiye. $\Vert {\mathbf{u}}_2\Vert =\sqrt{0+1+1}=\sqrt{2}$, toh ${\mathbf{q}}_2=\frac{1}{\sqrt{2}}\left[\begin{array}{c}0\\ 1\\ 1\end{array}\right]=\left[\begin{array}{c}0\\ 1/\sqrt{2}\\ 1/\sqrt{2}\end{array}\right].$ Check: ${\mathbf{q}}_2\cdot {\mathbf{q}}_2=0+\frac{1}{2}+\frac{1}{2}=1.$ ✓

Deciding question yeh hai: kya pair orthogonal hai? Shortcut $c_j=\frac{\mathbf{y}\cdot {\mathbf{u}}_j}{{\mathbf{u}}_j\cdot {\mathbf{u}}_j}$ orthogonality se derive hota hai — yeh ek coefficient isliye isolate karta hai kyunki saare cross-terms ${\mathbf{u}}_i\cdot {\mathbf{u}}_j$ ($i\ne j$) vanish ho jaate hain.

• ${\mathbf{v}}_1\cdot {\mathbf{v}}_2=1-1=0$ → orthogonal, shortcut valid.
• ${\mathbf{w}}_1\cdot {\mathbf{w}}_2=1+0=1\ne 0$ → orthogonal nahi, shortcut invalid; system solve karna padega.

$\mathbf{v}$-basis mein coordinates:

• $c_1=\frac{\mathbf{y}\cdot {\mathbf{v}}_1}{{\mathbf{v}}_1\cdot {\mathbf{v}}_1}=\frac{4+2}{2}=3.$
• $c_2=\frac{\mathbf{y}\cdot {\mathbf{v}}_2}{{\mathbf{v}}_2\cdot {\mathbf{v}}_2}=\frac{4-2}{2}=1.$

Check: $3{\mathbf{v}}_1+1{\mathbf{v}}_2=\left[\begin{array}{c}3+1\\ 3-1\end{array}\right]=\left[\begin{array}{c}4\\ 2\end{array}\right]=\mathbf{y}.$ ✓ Answer: $(3,1)$.

Lo ${\mathbf{u}}_1=\left[\begin{array}{c}1\\ 0\end{array}\right]$ aur ${\mathbf{u}}_2=\left[\begin{array}{c}0\\ 0\end{array}\right]=0$. Orthogonal? ${\mathbf{u}}_1\cdot {\mathbf{u}}_2=0$ ✓ (zero vector har cheez ke perpendicular hota hai, kyunki har product term $0$ hai). Independent? Nahi: $0\cdot {\mathbf{u}}_1+5\cdot {\mathbf{u}}_2=0$ ek nontrivial relation hai (coefficient $5\ne 0$) jo phir bhi zero sum deta hai — kyunki ${\mathbf{u}}_2$ kuch contribute hi nahi karta. Toh set dependent hai. Proof mein "nonzero" kyun chahiye: independence argument $c_j\Vert {\mathbf{u}}_j{\Vert }^2=0$ par khatam hoti hai. Agar ${\mathbf{u}}_j=0$ toh $\Vert {\mathbf{u}}_j{\Vert }^2=0$, aur $c_j\cdot 0=0$ kisi bhi $c_j$ ke liye hold karta hai — yeh $c_j=0$ force nahi kar sakta. Picture: zero-length arrow kahin bhi point nahi karta, isliye koi independent direction carry nahi karta.

(a) Normalize karo. Lengths: $\Vert {\mathbf{a}}_1\Vert =\sqrt{1+1}=\sqrt{2}$, $\Vert {\mathbf{a}}_2\Vert =\sqrt{2}$, $\Vert {\mathbf{a}}_3\Vert =\sqrt{4}=2$. ${\mathbf{q}}_1=\frac{1}{\sqrt{2}}\left[\begin{array}{c}1\\ 1\\ 0\end{array}\right],{\mathbf{q}}_2=\frac{1}{\sqrt{2}}\left[\begin{array}{c}1\\ -1\\ 0\end{array}\right],{\mathbf{q}}_3=\left[\begin{array}{c}0\\ 0\\ 1\end{array}\right].$ (b) Coordinates — denominators ab $1$ hain, toh sirf dot karo:

• $c_1=\mathbf{y}\cdot {\mathbf{q}}_1=\frac{1}{\sqrt{2}}(5+1+0)=\frac{6}{\sqrt{2}}=3\sqrt{2}.$
• $c_2=\mathbf{y}\cdot {\mathbf{q}}_2=\frac{1}{\sqrt{2}}(5-1+0)=\frac{4}{\sqrt{2}}=2\sqrt{2}.$
• $c_3=\mathbf{y}\cdot {\mathbf{q}}_3=4.$

Check (reassemble karo): $3\sqrt{2}{\mathbf{q}}_1=3\sqrt{2}\cdot \frac{1}{\sqrt{2}}(1,1,0)=(3,3,0)$; $2\sqrt{2}{\mathbf{q}}_2=(2,-2,0)$; $4{\mathbf{q}}_3=(0,0,4)$. Sum $=(5,1,4)=\mathbf{y}.$ ✓

$U=\left[\begin{array}{ccc}1/\sqrt{2}& 1/\sqrt{2}& 0\\ 1/\sqrt{2}& -1/\sqrt{2}& 0\\ 0& 0& 1\end{array}\right]$. $U\mathbf{x}=\left[\begin{array}{c}2/\sqrt{2}+0+0\\ 2/\sqrt{2}+0+0\\ 3\end{array}\right]=\left[\begin{array}{c}\sqrt{2}\\ \sqrt{2}\\ 3\end{array}\right]$. $\Vert U\mathbf{x}{\Vert }^2=2+2+9=13$. Aur $\Vert \mathbf{x}{\Vert }^2=4+0+9=13$. Equal ✓. Aisa kyun hona tha: $\Vert U\mathbf{x}{\Vert }^2=(U\mathbf{x})\cdot (U\mathbf{x})={\mathbf{x}}^{\mathsf{T}}{U}^{\mathsf{T}}U\mathbf{x}={\mathbf{x}}^{\mathsf{T}}I\mathbf{x}=\Vert \mathbf{x}{\Vert }^2$, $U^{\mathsf{T}}U=I$ use karke (orthonormal columns). Geometrically $U$ ek rotation/reflection hai — yeh arrow ko turn karta hai, kabhi stretch nahi karta.

Proof. Dot product ki bilinearity use karke length expand karo: $\Vert \mathbf{y}{\Vert }^2=\mathbf{y}\cdot \mathbf{y}=\left({\sum }_i{c}_i{\mathbf{q}}_i\right)\cdot \left({\sum }_j{c}_j{\mathbf{q}}_j\right)={\sum }_i{\sum }_j{c}_i{c}_j({\mathbf{q}}_i\cdot {\mathbf{q}}_j).$ Double sum kyun collapse hota hai: orthonormality ka matlab hai ${\mathbf{q}}_i\cdot {\mathbf{q}}_j=0$ jab tak $i=j$ nahi, aur $=1$ jab $i=j$. Har off-diagonal term khatam ho jaata hai; har diagonal term $c_i{c}_i\cdot 1$ rakhta hai: $\Vert \mathbf{y}{\Vert }^2={\sum }_i{c}_i^2.■$ Yeh $p$ dimensions mein Pythagoras hai: coordinates right-angled box ke legs hain, aur $\Vert \mathbf{y}\Vert$ diagonal hai. L4.1 par verify karo: $c_1^2+c_2^2+c_3^2=(3\sqrt{2}{)}^2+(2\sqrt{2}{)}^2+4^2=18+8+16=42$. Directly $\Vert \mathbf{y}{\Vert }^2=5^2+1^2+4^2=25+1+16=42.$ ✓

Yeh formula kyun: $W$ ka orthonormal basis hone par, $W$ ka $\mathbf{y}$ ke sabse kareeb point dotting se milta hai — wohi sasta trick, ab project karne ke liye use ho raha hai, exact coordinates dene ki jagah.

• $\mathbf{y}\cdot {\mathbf{q}}_1=\frac{1}{\sqrt{2}}(4+2+0)=\frac{6}{\sqrt{2}}=3\sqrt{2}.$
• $\mathbf{y}\cdot {\mathbf{q}}_2=5.$ $\hat{\mathbf{y}}=3\sqrt{2}\cdot \frac{1}{\sqrt{2}}\left[\begin{array}{c}1\\ 1\\ 0\end{array}\right]+5\left[\begin{array}{c}0\\ 0\\ 1\end{array}\right]=\left[\begin{array}{c}3\\ 3\\ 0\end{array}\right]+\left[\begin{array}{c}0\\ 0\\ 5\end{array}\right]=\left[\begin{array}{c}3\\ 3\\ 5\end{array}\right].$ Error: $\mathbf{y}-\hat{\mathbf{y}}=\left[\begin{array}{c}4-3\\ 2-3\\ 5-5\end{array}\right]=\left[\begin{array}{c}1\\ -1\\ 0\end{array}\right]$. $W$ ke perpendicular? Har basis vector ke saath dot karo:
• $(\mathbf{y}-\hat{\mathbf{y}})\cdot {\mathbf{q}}_1=\frac{1}{\sqrt{2}}(1-1+0)=0.$ ✓
• $(\mathbf{y}-\hat{\mathbf{y}})\cdot {\mathbf{q}}_2=0+0+0=0.$ ✓

Error dono spanning vectors ke orthogonal hai, isliye poore $W$ ke bhi — toh $\hat{\mathbf{y}}$ genuinely sabse kareeb point hai. Yeh Least squares ka seed hai aur, ${\mathbf{q}}_j$ ko columns mein stack karke, QR decomposition ka bhi.

L1 saare pairs check karo ::: L2 coordinate $=\frac{\mathbf{y}\cdot {\mathbf{u}}_j}{{\mathbf{u}}_j\cdot {\mathbf{u}}_j}$, denominator rakhna ::: L3 shortcut ko orthogonality & nonzero vectors chahiye ::: L4 orthonormal ⇒ denominator drop karo & lengths preserved ::: L5 $\Vert \mathbf{y}{\Vert }^2=\sum c_j^2$ aur dotting se projection.