Symmetric matrices — spectral theorem (real eigenvalues, orthogonal eigenvectors): Exercises

4.5.38 · D4 · Maths › Linear Algebra (Full) › Symmetric matrices — spectral theorem (real eigenvalues, ort

Reminders ek jagah, taaki koi bhi symbol tumhe achanak na mile:

Ek matrix symmetric hoti hai agar $A = A^{⊤}$ ho — isko apne main diagonal ke across flip karo (top-left se bottom-right tak) toh kuch nahi badalta. Neche mirror wali picture dekho.
$A^{⊤}$ ("A transpose") ka matlab hai "rows aur columns swap karo": row $i$ , column $j$ wali entry row $j$ , column $i$ par chali jaati hai.
$A$ ka eigenvector $x$ ek non-zero arrow hai jise $A$ sirf stretch karta hai, ghumata nahi: $A x = λ x$ . Stretch factor $λ$ ("lambda") eigenvalue hai. Dekho Eigenvalues and Eigenvectors.
Do arrows orthogonal (perpendicular) hote hain jab unka dot product zero ho: $u \cdot v = u_{1} v_{1} + u_{2} v_{2} + \dots = 0$ .
$Q$ orthogonal hota hai jab uske columns perpendicular unit arrows hon, jisse $Q^{⊤} Q = I$ aur isliye $Q^{- 1} = Q^{⊤}$ force hota hai. Dekho Orthogonal Matrices.

Neche wali figure — abhi padho, ye poori jagah repeat hoti hai. Left panel dikhata hai ki "symmetric" kaisi lagti hai: dashed mirror line ke dono taraf amber pair force equal hoti hai, aur yahi condition hai jo Exercise 1.1 test karta hai. Right panel woh payoff dikhata hai jo Exercise 3.1 mein prove hota hai: do eigen-axes ek clean $9 0^{\circ}$ par milte hain aur $A$ sirf unke saath stretch karta hai (cyan $3$ se, amber $1$ se) — koi twisting nahi. Ye picture dimag mein rakho; L2, L3 aur L5 sab iske andar hi hain.

Figure — Symmetric matrices — spectral theorem (real eigenvalues, orthogonal eigenvectors)

Level 1 — Recognition

Exercise 1.1

In mein se kaun si matrices symmetric hain? $A = (1225), B = (0 - 3 30), C = 410147074 .$

Recall Solution

Hum kya karte hain: har matrix ko uske apne transpose se compare karo — diagonal ke across mirror karo (bilkul upar wali figure mein dashed line ki tarah).

$A$ : off-diagonal entries hain $2$ (row 1) aur $2$ (transpose ki row 1) — match karte hain. Symmetric ✓.
$B$ : entry $(1, 2) = 3$ hai lekin entry $(2, 1) = - 3$ hai. Mirror karne se sign flip ho jaata hai, toh $B^{⊤} = - B$ . Not symmetric (ye antisymmetric hai).
$C$ : har mirror pair check karo: $(1, 2) = 1 = (2, 1)$ , $(1, 3) = 0 = (3, 1)$ , $(2, 3) = 7 = (3, 2)$ . Sab match karte hain. Symmetric ✓.

Answer: $A$ aur $C$ symmetric hain; $B$ nahi hai.

Exercise 1.2

$A = (2112)$ ke liye, verify karo ki $x = (1, 1)$ ek eigenvector hai aur uski eigenvalue nikalo.

Recall Solution

Hum kya karte hain: $A x$ multiply karo aur dekho ki output, input arrow ka scalar multiple hai ya nahi. $A x = (2112) (11) = (2 \cdot 1 + 1 \cdot 1 1 \cdot 1 + 2 \cdot 1) = (33) = 3 (11) .$ Output same direction mein point karta hai jaise $(1, 1)$ , bas $3 \times$ lamba. Toh $x = (1, 1)$ ek eigenvector hai eigenvalue $λ = 3$ ke saath.

Level 2 — Application

Exercise 2.1

$A = (3113)$ ko fully orthogonally diagonalize karo: $Q$ , $Λ$ nikalo jisme $A = Q Λ Q^{⊤}$ ho.

Recall Solution

Step 1 — eigenvalues. Kyun: eigenvalues stretch factors hain; dekho kahan $A - λ I$ kisi arrow ko zero par collapse karta hai, yaani kahan $det (A - λ I) = 0$ . $det (3 - λ 1 1 3 - λ) = (3 - λ)^{2} - 1 = λ^{2} - 6 λ + 8 = (λ - 4) (λ - 2) .$ Toh $λ = 4$ aur $λ = 2$ — dono real (jaisa Spectral Theorem promise karta hai).

Step 2 — eigenvectors. $(A - λ I) x = 0$ solve karo.

$λ = 4$ : $(- 1 1 1 - 1) x = 0 \Rightarrow - x_{1} + x_{2} = 0 \Rightarrow x = (1, 1)$ .
$λ = 2$ : $(1111) x = 0 \Rightarrow x_{1} + x_{2} = 0 \Rightarrow x = (1, - 1)$ .

Step 3 — orthogonality check. $(1, 1) \cdot (1, - 1) = 1 - 1 = 0$ ✓ (distinct eigenvalues ⇒ perpendicular — opening figure ka right panel, inhi exact axes ke saath).

Step 4 — normalize (length $1^{2} + 1^{2} = 2$ se divide karo taaki columns unit arrows hon, jisse $Q$ orthogonal bane): $Q = \frac{1}{2} (11 1 - 1), Λ = (4002) .$ Phir $A = Q Λ Q^{⊤}$ . Dekho Diagonalization.

Exercise 2.2

Exercise 2.1 ke data ko use karke, $A$ ko spectral sum $λ_{1} q_{1} q_{1}^{⊤} + λ_{2} q_{2} q_{2}^{⊤}$ ke roop mein likho aur confirm karo ki ye $A$ reproduce karta hai.

Recall Solution

Hum kya karte hain: har $q_{i} q_{i}^{⊤}$ eigen-axis $i$ par ek rank-1 projector hai; $λ_{i}$ se scale karo aur add karo. $q_{1} = \frac{1}{2} (1, 1)$ , $q_{2} = \frac{1}{2} (1, - 1)$ ke saath: $q_{1} q_{1}^{⊤} = \frac{1}{2} (1111), q_{2} q_{2}^{⊤} = \frac{1}{2} (1 - 1 - 1 1) .$ $4 \cdot \frac{1}{2} (1111) + 2 \cdot \frac{1}{2} (1 - 1 - 1 1) = (2222) + (1 - 1 - 1 1) = (3113) = A ✓$

Level 3 — Analysis

Exercise 3.1

Scratch se dikhao ki agar $A$ symmetric hai aur $u, v$ eigenvectors hain alag eigenvalues $λ \neq = μ$ ke saath, toh $u \cdot v = 0$ hota hai. (Koi numbers nahi — ek proof.)

Recall Solution

Idea: single number $u^{⊤} A v$ ko do alag tareekon se compute karo aur compare karo. Ye classic "ek quantity ko do tareekon se evaluate karo" trick hai.

Way 1 — $A$ ko $v$ par act karne do: $u^{⊤} A v = u^{⊤} (μ v) = μ (u^{⊤} v) .$

Way 2 — symmetry use karke $A$ ko $u$ par slide karo. Kyunki $u^{⊤} A = (A^{⊤} u)^{⊤}$ aur $A^{⊤} = A$ : $u^{⊤} A v = (A u)^{⊤} v = (λ u)^{⊤} v = λ (u^{⊤} v) .$

Compare: $μ (u^{⊤} v) = λ (u^{⊤} v) \Rightarrow (λ - μ) (u^{⊤} v) = 0$ . Kyunki $λ \neq = μ$ hai toh pehla factor non-zero hai, isliye $u^{⊤} v = 0$ , yaani $u ⊥ v$ . ∎ (Yahan $u^{⊤} v$ bas dot product hai jo matrix product ki tarah likha gaya hai.)

Exercise 3.2

Matrix $M = (1012)$ not symmetric hai. Iske eigenvectors hain $(1, 0)$ ( $λ = 1$ ke liye) aur $(1, 1)$ ( $λ = 2$ ke liye). Kya ye orthogonal hain? Ye kya reveal karta hai?

Recall Solution

Check: $(1, 0) \cdot (1, 1) = 1 \cdot 1 + 0 \cdot 1 = 1 \neq = 0$ . Not orthogonal. Ye kya reveal karta hai: eigenvectors ki orthogonality symmetry ka gift hai, koi universal law nahi. Exercise 3.1 ka proof "Way 2" mein $A^{⊤} = A$ use karta tha; us hypothesis ko hata do aur dono ways agree nahi karte, toh kuch bhi $u \cdot v = 0$ force nahi karta. Ek non-symmetric matrix phir bhi diagonalize ho sakti hai ( $M = P Λ P^{- 1}$ ), lekin $P^{- 1} \neq = P^{⊤}$ .

Level 4 — Synthesis

Exercise 4.1

Ek symmetric matrix repeated eigenvalue ke saath: $A = 500020002$ . Saare eigenvalues nikalo; phir, repeated eigenvalue ko carefully treat karte hue, ek full orthonormal eigenbasis produce karo, repeated eigenspace ke andar Gram–Schmidt run karte hue jab jo raw basis tum pick karo wo pehle se perpendicular na ho.

Recall Solution

Step 1 — eigenvalues. $A$ diagonal hai, toh iske eigenvalues uske diagonal entries hain: $λ = 5$ ek baar, aur $λ = 2$ multiplicity 2 ke saath. Ye exactly repeated-eigenvalue case hai.

Step 2 — simple eigenvalue $λ = 5$ . $(A - 5 I) x = 0$ solve karne par $y = z = 0$ force hota hai, jisse eigenvector $(1, 0, 0)$ milta hai — already unit length.

Step 3 — repeated eigenvalue $λ = 2$ (interesting case). $(A - 2 I) x = 0$ solve karo: $300000000 x = 0 \Rightarrow x = 0, y, z free .$ Eigenspace poora $y$ – $z$ plane hai — ek 2-dimensional space (symmetric matrices kabhi defective nahi hoti, toh eigenspace ki full dimension hoti hai). Maan lo humne jo do raw basis vectors pick kiye wo pehle se orthogonal nahi hain, jaise $w_{1} = (0, 1, 0), w_{2} = (0, 1, 1) (w_{1} \cdot w_{2} = 1 \neq = 0) .$ Gram–Schmidt eigenspace ke andar isko fix karta hai (Gram-Schmidt Process): $e_{1} = \frac{w _{1}}{∥ w _{1} ∥} = (0, 1, 0) .$ $w_{2}$ mein se $e_{1}$ ke along uski shadow subtract karo: $w_{2}^{⊥} = w_{2} - (w_{2} \cdot e_{1}) e_{1} = (0, 1, 1) - (1) (0, 1, 0) = (0, 0, 1),$ phir normalize karo (already unit hai): $e_{2} = (0, 0, 1)$ . Ab $e_{1} \cdot e_{2} = 0$ ✓.

Step 4 — full orthonormal eigenbasis assemble karo. $λ = 5$ vector ko do orthonormalized $λ = 2$ vectors ke saath combine karo: ${(1, 0, 0), (0, 1, 0), (0, 0, 1)} .$ $Q = 100010001, Λ = 500020002, A = Q Λ Q^{⊤} .$ Moral: ek distinct eigenvalue tumhe apna eigen-axis free mein deta hai; ek repeated eigenvalue tumhe ek poora plane deta hai, aur uske andar tumhe Gram–Schmidt se haath se orthonormal basis banani padti hai.

Exercise 4.2

Maan lo $A$ symmetric hai spectral decomposition $A = \sum_{i} λ_{i} q_{i} q_{i}^{⊤}$ ke saath. Prove karo ki $A^{2} = \sum_{i} λ_{i}^{2} q_{i} q_{i}^{⊤}$ , aur isko use karke $A^{k}$ state karo.

Recall Solution

Key fact: $q_{i}$ orthonormal hain, toh $q_{i}^{⊤} q_{j} = 1$ agar $i = j$ ho aur $0$ otherwise. $A = Q Λ Q^{⊤}$ se square karo: $A^{2} = (Q Λ Q^{⊤}) (Q Λ Q^{⊤}) = Q Λ (Q^{⊤} Q) Λ Q^{⊤} = Q Λ I Λ Q^{⊤} = Q Λ^{2} Q^{⊤} .$ Beech ka $Q^{⊤} Q = I$ collapse ho jaata hai (yahi orthogonality kaam karti hai). Kyunki $Λ^{2}$ diagonal hai entries $λ_{i}^{2}$ ke saath: $A^{2} = \sum_{i} λ_{i}^{2} q_{i} q_{i}^{⊤} .$ General power: $A^{k} = Q Λ^{k} Q^{⊤} = \sum_{i} λ_{i}^{k} q_{i} q_{i}^{⊤}$ . Eigen-axes frozen hain; sirf stretch factors power par raise hote hain.

Level 5 — Mastery

Exercise 5.1

$A = (3113)$ ke liye (Exercise 2.1 se, eigenvalues $4, 2$ ), quadratic form $q (x) = x^{⊤} A x$ ek curve trace karta hai jab $q (x) = 1$ set kiya jaaye. Eigen-axes use karke, shape identify karo, uski axis directions, aur uski semi-axis lengths.

Recall Solution

Eigen-coordinates mein rotate karo. $x = Q y$ rakho $Q$ ke saath Exercise 2.1 se. Toh $q = x^{⊤} A x = (Q y)^{⊤} (Q Λ Q^{⊤}) (Q y) = y^{⊤} Λ y = 4 y_{1}^{2} + 2 y_{2}^{2} .$ Cross term $2 x_{1} x_{2}$ vanish ho gaya — exactly yahi eigen-axes tumhe dete hain. Dekho Quadratic Forms. $= 1$ set karo: $4 y_{1}^{2} + 2 y_{2}^{2} = 1$ , yaani $\frac{y _{1}^{2}}{( 1/2 ) ^{2}} + \frac{y _{2}^{2}}{( 1/ 2 ) ^{2}} = 1$ . Ye ek ellipse hai (neche wali figure mein draw ki gayi hai).

$q_{1} = \frac{1}{2} (1, 1)$ ke along semi-axis ki length $1/ λ_{1} = 1/ 4 = \frac{1}{2}$ hai.
$q_{2} = \frac{1}{2} (1, - 1)$ ke along semi-axis ki length $1/ λ_{2} = 1/ 2 \approx 0.707$ hai.

Original $x$ –coordinates mein axis directions: ellipse ka short axis $q_{1} = \frac{1}{2} (1, 1)$ ke along point karta hai ( $4 5^{\circ}$ line), aur uska long axis $q_{2} = \frac{1}{2} (1, - 1)$ ke along ( $- 4 5^{\circ}$ line). Ye exactly eigen-axes hain — ellipse $4 5^{\circ}$ par tilted hai, $x$ – aur $y$ –axes ke aligned nahi.

Isko padhna: bada eigenvalue ⇒ steeper form ⇒ shorter semi-axis, toh $λ_{1} = 4$ short axis $\frac{1}{2}$ deta hai aur $λ_{2} = 2$ long axis $1/ 2$ deta hai. Dono eigenvalues positive hain ⇒ curve ek ellipse mein close hoti hai ⇒ $A$ positive definite hai.

Exercise 5.2

Maan lo $A = (2112)$ . Iske singular values wo numbers hain jo SVD mein appear karte hain. Explain karo kyun, is $A$ ke liye, singular values $∣ λ_{1} ∣, ∣ λ_{2} ∣$ ke barabar hain, aur unhe numerically do.

Recall Solution

Eigenvalues (parent Example 1): $λ = 3, 1$ . Singular values $eigenvalues of A^{⊤} A$ hain. Symmetric $A$ ke liye, $A^{⊤} A = A^{2}$ , jiske eigenvalues $λ_{i}^{2}$ hain. Toh singular values $= λ_{i}^{2} = ∣ λ_{i} ∣$ . Yahan $∣3∣ = 3$ , $∣1∣ = 1$ , toh singular values $3 and 1$ hain. Subtlety: ye $∣ λ_{i} ∣$ , yaani absolute values ke barabar hain. Agar koi eigenvalue negative hoti, toh uska singular value uska magnitude hota — sign SVD ke orthogonal factors mein absorb ho jaata hai.

Exercise 5.3

Maan lo symmetric $A$ ke eigenvalues $λ = 4$ (eigenvector $\frac{1}{2} (1, 1)$ ) aur $λ = - 2$ (eigenvector $\frac{1}{2} (1, - 1)$ ) hain. $A$ ko uske spectral sum se reconstruct karo.

Recall Solution

$A = 4 \cdot \frac{1}{2} (1111) + (- 2) \cdot \frac{1}{2} (1 - 1 - 1 1) = (2222) + (- 1 1 1 - 1) = (1331) .$ Check: trace $= 2 = 4 + (- 2)$ ✓, $det = 1 - 9 = - 8 = 4 \cdot (- 2)$ ✓. Negative eigenvalue $A$ ko indefinite banati hai — quadratic form $q (x) = 1$ ek hyperbola hogi, ellipse nahi.

Flashcards

Real symmetric matrix ke liye, distinct eigenvalues ke eigenvectors orthogonal kyun hone chahiye?

u^{⊤} A v

ko do tareekon se compute karne par

(λ - μ) (u^{⊤} v) = 0

milta hai; distinct

λ \neq = μ

force karta hai

u^{⊤} v = 0

Symmetric matrix ke singular values uske eigenvalues ke terms mein kya hote hain?

Absolute values

∣ λ_{i} ∣

Symmetric

A = Q Λ Q^{⊤}

ke liye,

A^{k}

kya hai?

A^{k} = Q Λ^{k} Q^{⊤} = \sum_{i} λ_{i}^{k} q_{i} q_{i}^{⊤}

— same axes, powered stretches.