Visual walkthrough — Orthogonal sets and orthonormal basis

4.5.34 · D2 · Maths › Linear Algebra (Full) › Orthogonal sets and orthonormal basis

Yeh page Orthogonal sets and orthonormal basis ke peeche ki picture-by-picture derivation hai. Jahan parent note ne coordinate formula bataya tha, yahan hum use kamate hain, ek drawing at a time.

---

Step 1 — Vector kya hota hai, aur "dot" kya measure karta hai?

KYA. Vector sirf origin (point $(0,0)$) se kisi jagah tak ka arrow hota hai. Hum $\mathbf{y}$ bold mein likhte hain matlab "poora arrow", aur uski tip ke coordinates ek column mein, e.g. $\mathbf{y}=\left[\begin{array}{c}4\\ 3\end{array}\right]$ matlab "4 right jao, 3 upar jao."

DOT PRODUCT kyun. Hum ek aisi machine dhundh rahe hain jo do arrows khaaye aur ek single number nikale jo bataye woh kitna same direction mein point karte hain. Woh machine hai dot product. Do arrows $\mathbf{a}=\left[\begin{array}{c}{a}_1\\ a_2\end{array}\right]$ aur $\mathbf{b}=\left[\begin{array}{c}{b}_1\\ b_2\end{array}\right]$ ke liye:

$\underset{\text{one number out}}{\underbrace{\mathbf{a}\cdot \mathbf{b}}}=\underset{\text{a’s }x}{\underbrace{a_1}}\underset{\text{b’s }x}{\underbrace{b_1}}+\underset{\text{a’s }y}{\underbrace{a_2}}\underset{\text{b’s }y}{\underbrace{b_2}}$

Matching slots ko multiply karo, phir add karo. Is same number ka ek aur chehra hai jo hume baar baar chahiye:

$\mathbf{a}\cdot \mathbf{b}=\underset{\text{length of }a}{\underbrace{\Vert \mathbf{a}\Vert }}\underset{\text{length of }b}{\underbrace{\Vert \mathbf{b}\Vert }}\underset{\text{how aligned}}{\underbrace{\cos \theta }}$

Yahan $\Vert \mathbf{a}\Vert$ arrow ki length hai (Pythagoras: $\Vert \mathbf{a}\Vert =\sqrt{a_1^2+a_2^2}$), aur $\theta$ do arrows ke beech ka angle hai. Jo key fact hum use karenge: jab arrows perpendicular hote hain, $\theta =90^{\circ }$, aur $\cos 90^{\circ }=0$, toh poora dot product $0$ ho jaata hai.

PICTURE. Do arrows aur unke beech ka angle; dot number $0$ ki taraf shrink hota hai jab woh right angle ki taraf swing karte hain.

---

Step 2 — "Orthogonal basis" kaisa dikhta hai

KYA. Ek basis arrows ka ek set hai jo hum axes ki tarah use karte hain — har point ko "har axis ke along kitna" se describe karte hain. Ek orthogonal basis mein axes mutually perpendicular hone zaroori hain. Unhe ${\mathbf{u}}_1,{\mathbf{u}}_2,\dots$ kahte hain. Perpendicular matlab, Step 1 se,

${\mathbf{u}}_i\cdot {\mathbf{u}}_j=0\text{for }i\ne j(\text{different axes}\Rightarrow \text{dot is }0)$

YEH KYUN. Ordinary axes kisi bhi angle par jhuk sakti hain. Jab woh jhukti hain, coordinates aapas mein entangle ho jaate hain aur tumhe equations solve karni padti hain. Jab woh clean right angle par baith jaati hain, har coordinate doosre se independent ho jaata hai — yahi woh payoff hai jise hum paa rahe hain.

PICTURE. Left: ek slanted generic basis (axes perpendicular nahi) — messy. Right: ek orthogonal basis (axes $90^{\circ }$ par) — clean.

---

Step 3 — Goal: $\mathbf{y}$ ko axes ki recipe ki tarah likhna

KYA. Koi bhi target arrow $\mathbf{y}$ lo jo hamare axes ke span mein rehta ho. Kyunki woh basis form karte hain, kuch amounts $c_1,c_2$ exist karte hain taaki

$\mathbf{y}=\underset{\text{amount of }{u}_1}{\underbrace{c_1}}{\mathbf{u}}_1+\underset{\text{amount of }{u}_2}{\underbrace{c_2}}{\mathbf{u}}_2$

YEH SIRF AADHA JAWAB KYUN HAI. Hum jaante hain $c$ exist karte hain, lekin unki values abhi nahi jaante. Ek slanted basis ke liye, $c_1$ aur $c_2$ nikalte hain matlab do coupled equations solve karna. Hamara poora mission: dikhao ki perpendicularity hume har $c_j$ akele nikalte deti hai.

PICTURE. $\mathbf{y}$ ko $c_1{\mathbf{u}}_1$ aur $c_2{\mathbf{u}}_2$ se bane parallelogram ki diagonal ke roop mein draw kiya gaya.

---

Step 4 — Magic move: dono sides ko ek axis se dot karo

KYA. Step 3 ki equation lo aur sab kuch ko ek chosen axis ${\mathbf{u}}_1$ se dot karo:

$\mathbf{y}\cdot {\mathbf{u}}_1=(c_1{\mathbf{u}}_1+c_2{\mathbf{u}}_2)\cdot {\mathbf{u}}_1=c_1({\mathbf{u}}_1\cdot {\mathbf{u}}_1)+c_2(\underset{=0\text{by Step 2}}{\underbrace{{\mathbf{u}}_2\cdot {\mathbf{u}}_1}})$

YEH EXACT TOOL KYUN. Dot product woh ek hi operation hai jo "axis se poochh sake ki usne kitna dekha." Kyunki ${\mathbf{u}}_2\perp {\mathbf{u}}_1$, cross term ${\mathbf{u}}_2\cdot {\mathbf{u}}_1$ exactly zero hai — $c_2$ ka contribution gayab ho jaata hai. Ek perpendicularity fact ne abhi ek unknown delete kar diya. Jo bachta hai woh $c_1$ mein sirf ek clean equation hai:

$\mathbf{y}\cdot {\mathbf{u}}_1=c_1({\mathbf{u}}_1\cdot {\mathbf{u}}_1)$

PICTURE. $c_2{\mathbf{u}}_2$ piece ko ${\mathbf{u}}_1$ ke perpendicular draw kiya gaya hai; ${\mathbf{u}}_1$ line par uska "shadow" length $0$ hai — yeh $\mathbf{y}\cdot {\mathbf{u}}_1$ mein kuch contribute nahi karta.

---

Step 5 — Coordinate solve karo (yeh decouple ho jaata hai!)

KYA. Survivor equation ko ${\mathbf{u}}_1\cdot {\mathbf{u}}_1$ number se divide karo:

$c_1=\frac{\stackrel{\text{how much }y\text{ leans along }{u}_1}{\overbrace{\mathbf{y}\cdot {\mathbf{u}}_1}}}{\underset{=\Vert {\mathbf{u}}_1{\Vert }^2,\text{axis’s own size}}{\underbrace{{\mathbf{u}}_1\cdot {\mathbf{u}}_1}}}$

${\mathbf{u}}_2$ ke saath wahi argument $c_2=\frac{\mathbf{y}\cdot {\mathbf{u}}_2}{{\mathbf{u}}_2\cdot {\mathbf{u}}_2}$ deta hai, aur generally:

DENOMINATOR KYUN. ${\mathbf{u}}_j\cdot {\mathbf{u}}_j=\Vert {\mathbf{u}}_j{\Vert }^2$ measure karta hai ki axis $j$ kitni lambi hai. Numerator $\mathbf{y}\cdot {\mathbf{u}}_j$ mein axis ki length bhi mix hai, toh hum use wapis divide karte hain — warna ek lamba ruler galat count report karta. Geometrically yeh exactly $\mathbf{y}$ ka ${\mathbf{u}}_j$-line par Orthogonal projection hai.

PICTURE. $\mathbf{y}$ ki tip se ${\mathbf{u}}_1$ line par perpendicular giraao; foot $c_1{\mathbf{u}}_1$ mark karta hai. Numerator ki height divide karne ke baad axis ke along ek length ban jaati hai.

---

Step 6 — Orthonormal shortcut (denominators gayab ho jaate hain)

KYA. Agar hum pehle har axis ko length $1$ tak shrink kar lein (called normalizing: ${\mathbf{u}}_j$ ki jagah ${\mathbf{q}}_j={\mathbf{u}}_j/\Vert {\mathbf{u}}_j\Vert$ use karo), toh ${\mathbf{q}}_j\cdot {\mathbf{q}}_j=1$ aur denominator $1$ ban jaata hai:

$c_j=\mathbf{y}\cdot {\mathbf{q}}_j(\text{denominator }=1)$

KYUN. Kisi arrow ko uski apni length se divide karne par uski direction same rehti hai lekin uski length exactly $1$ ho jaati hai. Unit axes ke saath, "$\mathbf{y}$ kitna ${\mathbf{q}}_j$ ke along jhukta hai" wahi coordinate hai — koi rescaling nahi chahiye.

PICTURE. Ek length-$1$ axis jisme $\mathbf{y}$ ka shadow directly coordinate ke roop mein land karta hai.

---

Step 7 — Edge & degenerate cases (reader ko kabhi stranded mat chodo)

KYA / KYUN. Teen scenarios recipe ko tod sakte hain — yahan bataya hai ki har ek mein kya hota hai.

• Zero axis (${\mathbf{u}}_j=0$). Tab ${\mathbf{u}}_j\cdot {\mathbf{u}}_j=0$ aur hum zero se divide kar rahe hote. Isliye parent note insist karta hai ki vectors nonzero hon. Zero arrow kisi taraf point nahi karta aur axis nahi ho sakta.
• $\mathbf{y}$ ek axis ke perpendicular. Tab $\mathbf{y}\cdot {\mathbf{u}}_j=0$, toh $c_j=0$ — bilkul theek hai, matlab sirf yeh hai ki $\mathbf{y}$ ka us axis ke along koi component nahi. Failure nahi.
• $\mathbf{y}$ span ke bahar (axes dimensions se kam). Formula phir bhi chalega aur $\mathbf{y}$ ka span ke andar best approximation return karega — woh piece jo axes dekh sakti hain. Bacha hua perpendicular part har dot product ke liye invisible hai. Yeh Least squares ka seed hai.

PICTURE. Teen mini-panels: (a) ek zero axis crossed out, (b) $\mathbf{y}\perp {\mathbf{u}}_1$ giving $c_1=0$, (c) $\mathbf{y}$ ek line se bahar nikal raha, uska shadow (projection) plus ek red leftover.

---

Ek-picture summary

Upar sab kuch compress karke: $\mathbf{y}$, do perpendicular axes, do perpendicular drops jo $c_1$ aur $c_2$ dete hain, har ek apne dot-product formula ke saath labelled. Ise right-to-left padho jaise "shadow, divide, done."

Recall Feynman retelling — plain words mein wapis bolo

Imagine karo do rulers floor par ek perfect right angle par rakhe hue hain. Mere paas ek arrow hai jo kisi jagah point kar raha hai aur main jaanna chahta hoon ki woh har ruler ke along kitna door tak pahunchta hai. Trick: main ruler one par seedha light giraata hoon — shadow ki length batati hai mere arrow ka kitna hissa ruler one ke along hai, aur dot product $\mathbf{y}\cdot {\mathbf{u}}_1$ woh shadow compute karta hai. Ruler two $90^{\circ }$ par hai, toh woh ruler one par koi shadow nahi daalta — isliye do answers interfere nahi karte aur mujhe kabhi equations solve nahi karni padti. Mujhe ruler ki apni length (${\mathbf{u}}_1\cdot {\mathbf{u}}_1$) se divide karna padta hai taaki ek lamba ruler over-count na kare; agar main pehle har ruler ko length $1$ tak cut kar loon, woh division bhi gayab ho jaata hai. Zero-length rulers banned hain (divide by zero), length zero ka shadow matlab sirf yeh hai ki mera arrow us ruler ke perpendicular hai, aur agar mera arrow floor se bahar point karta hai, toh shadows mujhe sabse close in-floor arrow dete hain — baaki sight se kho jaata hai. Woh last fact hi least squares ka poora idea hai.

Prerequisites & next steps: Dot product and norms aur Orthogonal projection par bana hua; ek orthogonal basis manufacture karne ki machinery hai Gram-Schmidt process; orthonormal axes ko ek matrix mein package karna QR decomposition tak le jaata hai; "leftover" idea Least squares ko power karta hai; symmetric matrices free mein orthonormal axes deti hain Eigenvalues and eigenvectors ke zariye.