Visual walkthrough — Symmetric matrices — spectral theorem (real eigenvalues, orthogonal eigenvectors)

4.5.38 · D2 · Maths › Linear Algebra (Full) › Symmetric matrices — spectral theorem (real eigenvalues, ort

Shuru karne se pehle, teen words jo har figure pe aayenge. Inhe dheere se bolo, picture dekho, phir padhte raho.

---

Step 1 — Ek matrix har arrow ke saath kya karti hai

KYA. Symmetric matrix $A=\left(\begin{array}{cc}2& 1\\ 1& 2\end{array}\right)$ lo (notice karo $A=A^{\top }$: do off-diagonal $1$'s match karte hain). Unit circle lo — length $1$ ke har arrow, har direction mein — aur unme se har ek par $A$ apply karo.

KYUN. "$A$ space ke saath kya karta hai" ke baare mein ek theorem tabhi samajh aata hai jab hum pehle $A$ ko kaam karte dekhen. Hum abhi kuch derive nahi kar rahe; hum evidence collect kar rahe hain.

PICTURE. Blue unit circle orange ellipse ban jaata hai. Dhyan se dekho: zyaadatar input arrows output mein alag direction mein point karte hain jis direction mein gaye the — arrow twist ho gaya. Lekin do special directions (red aur green arrows) bilkul same direction mein output aate hain, bas lamba ya chhota. Woh eigenvectors hain.

$\underset{\text{machine}}{\underbrace{A}}\underset{\text{unit input}}{\underbrace{\mathbf{x}}}=\underset{\text{output on ellipse}}{\underbrace{A\mathbf{x}}}$

---

Step 2 — Special arrows ko naam dena: eigen-equation

KYA. Un do non-twisting directions ke liye, defining rule likho.

KYUN. Hume ek clean equation chahiye jis par poora proof tike. Har symbol apni jagah kamaata hai.

$\underset{\text{the machine}}{\underbrace{A}}\underset{\begin{array}{c}\text{special}\\ \text{direction}\end{array}}{\underbrace{\mathbf{x}}}=\underset{\begin{array}{c}\text{kitna}\\ \text{stretch hota hai}\end{array}}{\underbrace{\lambda }}\underset{\begin{array}{c}\text{same}\\ \text{direction}\end{array}}{\underbrace{\mathbf{x}}}$

Ise zor se padho: "$A$ jo $\mathbf{x}$ par act karta hai woh $\lambda$ times same $\mathbf{x}$ deta hai." Output input ki line par rehta hai. $\lambda$ bada $\Rightarrow$ lamba stretch; $\lambda$ negative $\Rightarrow$ arrow peeche ki taraf flip ho jaata hai.

PICTURE. Green arrow $\lambda =3$ se stretch hota hai (teen guna lamba, same direction). Red arrow barely badalta hai, $\lambda =1$ (same length, same direction). Koi rotate nahi hua — yahi poora point hai.

---

Step 3 — Kya $\lambda$ secretly ek imaginary number ho sakta hai?

KYA. Hume complex eigenvalues ko rule out karna hai, sirf hope nahi kar sakte ki woh real hain. Sawaal poochne ke liye bhi hume ek nayi tool chahiye.

YEH tool kyun, koi aur kyun nahi? Hume ek aisi quantity chahiye jise hum do alag tarizon se compute kar sakein aur real hone par majboor kar sakein. Length-squared guaranteed real aur positive hai — $\lambda$ ko trap karne ke liye perfect.

PICTURE. Ek complex arrow apne real aur imaginary parts ke roop mein draw kiya gaya hai; ${\mathbf{x}}^{\ast }\mathbf{x}$ unhe collapse karke ek honest positive length mein laata hai, ek filled bar ke roop mein dikhaaya gaya hai.

---

Step 4 — $\lambda$ ko trap karna: yeh real hona chahiye

KYA. Machine ko ${\mathbf{x}}^{\ast }$ aur $\mathbf{x}$ ke beech mein sandwich karo aur number ko do tariyon se compute karo.

Way 1 — right side par eigen-equation use karo: ${\mathbf{x}}^{\ast }A\mathbf{x}={\mathbf{x}}^{\ast }(\underset{=\lambda \mathbf{x}}{\underbrace{A\mathbf{x}}})=\lambda \underset{\text{real}>0}{\underbrace{({\mathbf{x}}^{\ast }\mathbf{x})}}.$

Way 2 — us single number ka conjugate transpose lo. Ek $1\times 1$ number apne conjugate transpose ke barabar hota hai, aur $A^{\ast }=A$ kyunki $A$ real aur symmetric hai: $({\mathbf{x}}^{\ast }A\mathbf{x}{)}^{\ast }={\mathbf{x}}^{\ast }\underset{=A}{\underbrace{A^{\ast }}}\mathbf{x}={\mathbf{x}}^{\ast }A\mathbf{x}\Rightarrow \text{number real hai.}$

Compare karo. Way 1 kehta hai number $\lambda ({\mathbf{x}}^{\ast }\mathbf{x})$ hai; conjugate karne par $\bar{\lambda }({\mathbf{x}}^{\ast }\mathbf{x})$ milta hai. Agar koi number apne conjugate ke barabar hai toh woh real hai, isliye: $(\lambda -\bar{\lambda })\underset{>0}{\underbrace{({\mathbf{x}}^{\ast }\mathbf{x})}}=0\Rightarrow \lambda =\bar{\lambda }\Rightarrow \boxed{\lambda \in \mathbb{R}}.$

KYUN. $\lambda =\bar{\lambda }$ literally matlab hai "imaginary part $=0$." Ek number jo apne mirror-in-the-real-axis ke barabar hai, real axis par hi hona chahiye.

PICTURE. Complex plane mein, $\lambda$ aur uska conjugate $\bar{\lambda }$ real axis ke across mirror images hain. Equation $\lambda =\bar{\lambda }$ unhe coincide hone par majboor karta hai — aur woh sirf real line par hi ho sakta hai.

---

Step 5 — Eigen-axes exactly $90°$ par kyun milte hain

KYA. Do eigenvectors $\mathbf{u},\mathbf{v}$ lo jinka alag eigenvalues $\lambda \ne \mu$ ho. Single number ${\mathbf{u}}^{\top }A\mathbf{v}$ ko do tariyon se compute karo.

Dot product ${\mathbf{u}}^{\top }\mathbf{v}=\sum u_i{v}_i$ alignment measure karta hai: yeh $0$ hota hai exactly tab jab arrows perpendicular hote hain. Yahi woh quantity hai jise hum prove karna chahte hain ki zero hai.

Way A — $A$ ko $\mathbf{v}$ par hit karne do: ${\mathbf{u}}^{\top }A\mathbf{v}={\mathbf{u}}^{\top }(\underset{A\mathbf{v}}{\underbrace{\mu \mathbf{v}}})=\mu ({\mathbf{u}}^{\top }\mathbf{v}).$

Way B — symmetry use karke $A$ ko $\mathbf{u}$ ki taraf slide karo ($A^{\top }=A$): ${\mathbf{u}}^{\top }A\mathbf{v}=(\underset{=A}{\underbrace{A^{\top }}}\mathbf{u}{)}^{\top }\mathbf{v}=(\lambda \mathbf{u}{)}^{\top }\mathbf{v}=\lambda ({\mathbf{u}}^{\top }\mathbf{v}).$

Equate karo: $\mu ({\mathbf{u}}^{\top }\mathbf{v})=\lambda ({\mathbf{u}}^{\top }\mathbf{v})$, toh $(\lambda -\mu )({\mathbf{u}}^{\top }\mathbf{v})=0.$ Kyunki $\lambda -\mu \ne 0$, doosra factor zero hona chahiye: ${\mathbf{u}}^{\top }\mathbf{v}=0$. Perpendicular. ∎

KYUN symmetry hero hai. Way B sirf isliye kaam karta hai kyunki $A^{\top }=A$ ne machine ko $\mathbf{v}$ se $\mathbf{u}$ par move karne diya. Symmetry hatao aur yeh bridge collapse ho jaata hai — isliye non-symmetric matrices ke eigenvectors skewed hote hain.

PICTURE. $\left(\begin{array}{cc}2& 1\\ 1& 2\end{array}\right)$ ke do eigen-directions, $(1,1)$ aur $(1,-1)$, ek clean right angle par milte hue draw kiye gaye hain, aur dot product $0$ par collapse hota hua dikhaya gaya hai.

---

Step 6 — Edge case: agar koi eigenvalue repeat kare toh?

KYA. Step 5 ki trick ko $\lambda \ne \mu$ chahiye tha. $\left(\begin{array}{ccc}2& 0& 0\\ 0& 3& 1\\ 0& 1& 3\end{array}\right)$ ke baare mein kya, jahan $\lambda =2$ do baar aata hai?

KYUN chinta karo. Ek general matrix ke liye, ek repeated eigenvalue "defective" ho sakta hai — bahut kam eigenvectors, koi full basis nahi. Hume dikhana hai ki symmetry yeh forbid karti hai.

Rescue. Ek symmetric matrix kabhi bhi defective nahi hota: ek $k$-fold eigenvalue hamesha ek full $k$-dimensional eigenspace ke saath aata hai — eigenvectors ka ek poora flat plane (ya aur zyaada). Alag eigenspaces ke koi bhi do vectors abhi bhi perpendicular hain (Step 5). Repeated plane ke andar tum koi bhi do vectors choose karne ke liye free ho; Gram-Schmidt Process run karo unhe orthonormal banane ke liye. Toh tum hamesha ek full perpendicular set assemble kar sakte ho.

PICTURE. $\lambda =2$ ke liye eigenspace ek poora plane hai; do chosen orthogonal arrows uske andar baithte hain, aur $\lambda =4$ eigenvector us plane ke perpendicular bahar nikalti hai.

---

Step 7 — Ise $A=Q\Lambda Q^{\top }$ mein pack karna

KYA. Unit eigenvectors ${\mathbf{q}}_1,\dots ,{\mathbf{q}}_n$ ko ek matrix $Q$ ke columns ke roop mein line up karo. Kyunki woh perpendicular aur length $1$ hain (orthonormal), $Q$ ek orthogonal matrix hai: $Q^{\top }Q=I$, jiska matlab hai $Q^{-1}=Q^{\top }$.

Saari eigen-equations side by side stack karo: $A\underset{[{\mathbf{q}}_1\cdots {\mathbf{q}}_n]}{\underbrace{Q}}=\underset{[{\lambda }_1{\mathbf{q}}_1\cdots {\lambda }_n{\mathbf{q}}_n]}{\underbrace{Q\Lambda }},\Lambda =\left(\begin{array}{ccc}{\lambda }_1& & \\ & \ddots & \\ & & {\lambda }_n\end{array}\right).$ Left side ka column $i$ hai $A{\mathbf{q}}_i$; right ka column $i$ hai ${\lambda }_i{\mathbf{q}}_i$ — exactly eigen-equation. Ab right mein $Q^{\top }=Q^{-1}$ se multiply karo: $\boxed{A=Q\Lambda Q^{\top }.}$

YEH $P\Lambda P^{-1}$ se better kyun hai. Koi bhi diagonalizable matrix $A=P\Lambda P^{-1}$ deta hai, lekin $P^{-1}$ compute karna kaam hai. Symmetry $P^{-1}$ ko free-of-charge transpose $Q^{\top }$ mein upgrade karti hai. Yahi orthogonality bonus hai.

PICTURE. $A=Q\Lambda Q^{\top }$ ko right-to-left padhte hue ek arrow ka teen-step safar: $Q^{\top }$ eigen-coordinates mein rotate karta hai, $\Lambda$ axes ke saath ${\lambda }_i$ se stretch karta hai, $Q$ wapas rotate karta hai. Koi bhi shear kabhi nahi hoti.

---

Ek-picture summary

Sab kuch ek canvas par: ek symmetric matrix unit circle ko ek ellipse mein le jaata hai jiske axes perpendicular eigenvectors hain, real eigenvalues se stretch hue — aur woh stretch-along-perpendicular-axes hi $A=Q\Lambda Q^{\top }$ hai.

Recall Feynman retelling — ise bina equations ke ek dost ko bolo

Ek symmetric matrix ek gentle machine hai: usse koi bhi arrow do aur woh stretch hokar wapas aata hai, kabhi sheared nahi, kabhi imaginary land mein spin nahi hota. Isse dekhne ke liye, unit circle ko ellipse bante dekho — ellipse ke apne do axes woh arrows hain jinhe machine twist karne se mana kar deti hai. Woh axes eigenvectors hain aur unke stretch factors eigenvalues hain. Do facts bahar aate hain aur dono seedhe mirror-symmetry $A=A^{\top }$ se aate hain. Pehla, stretch factors honest real numbers hain: machine ko ek length-squared ke andar sandwich karo (jo hamesha positive real hoti hai) aur yeh factor ko apne real axis ke across mirror image ke barabar hone par majboor karta hai — sirf real numbers hi aisa karte hain. Doosra, axes perfect right angle par cross karte hain: machine ko ek eigenvector se doosre par slide karo (sirf symmetry yeh slide karne deti hai), aur do alag stretch factors sirf tabhi agree kar sakte hain jab arrows ka dot product zero ho — perpendicular. Jab bhi koi stretch factor repeat karta hai, symmetry tumhe eigenvectors ka ek poora flat plane deti hai, toh tum unhe hamesha right-angle set mein straighten kar sakte ho. Un perpendicular unit arrows ko $Q$ ke columns ke roop mein bundle karo; phir machine literally hai "good axes mein rotate karo, stretch karo, wapas rotate karo" — woh sentence exactly $A=Q\Lambda Q^{\top }$ hai.