Eigenvalues and eigenvectors

1.1.13 · AI-ML › Linear Algebra Essentials

HUM POOCH KYA RAHE HAIN?

• Nonzero kyun? $v=0$ trivially $A\cdot 0=\lambda \cdot 0$ satisfy karta hai kisi bhi $\lambda$ ke liye, toh yeh kuch bhi nahi batata. Isliye hum ise forbid karte hain.
• ML mein $\lambda$ kyun matter karta hai? PCA, covariance structure, PageRank, gradient descent ki stability, spectral clustering — sab "special directions dhundho" wale problems hain.

INHE DHUNDHTE KAISE HAIN? (Scratch se Derivation)

Hume chahiye $Av=\lambda v$ with $v\ne 0$. Rearrange karo:

$Av-\lambda v=0\Rightarrow Av-\lambda Iv=0\Rightarrow (A-\lambda I)v=0$

Ab key logical step. $(A-\lambda I)v=0$ ek homogeneous system hai.

• Agar $(A-\lambda I)$ invertible hota, toh ek hi solution hota $v=(A-\lambda I{)}^{-1}0=0$. Lekin humne $v=0$ ban kiya hai.
• Toh nonzero $v$ ke exist hone ke liye, $(A-\lambda I)$ singular (non-invertible) hona chahiye.
• Ek matrix singular hota hai $⟺$ uska determinant zero hota hai.

Phir har $\lambda$ ke liye, $(A-\lambda I)v=0$ mein wapas plug karo aur null-space vectors $v$ solve karo.

Worked Example 1 — ek clean $2\times 2$

$A=\left(\begin{array}{cc}2& 1\\ 1& 2\end{array}\right)$

Step 1: $A-\lambda I$ banao. Kyun? Humne derive kiya ki eigenvalues is matrix ko singular banate hain. $A-\lambda I=\left(\begin{array}{cc}2-\lambda & 1\\ 1& 2-\lambda \end{array}\right)$

Step 2: determinant = 0. Kyun? Singular $⟺\det =0$. $(2-\lambda {)}^2-(1)(1)=0$ ${\lambda }^2-4\lambda +3=0\Rightarrow (\lambda -1)(\lambda -3)=0$ Toh ${\lambda }_1=1,{\lambda }_2=3$.

Step 3: $\lambda =3$ ka eigenvector. Kyun? Direction dhundhne ke liye $(A-\lambda I)v=0$ mein substitute karo. $\left(\begin{array}{cc}-1& 1\\ 1& -1\end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)=0\Rightarrow -x+y=0\Rightarrow y=x$ Eigenvector $v_1=\left(\begin{array}{c}1\\ 1\end{array}\right)$ (koi bhi scale).

Step 4: $\lambda =1$ ka eigenvector. $\left(\begin{array}{cc}1& 1\\ 1& 1\end{array}\right)v=0\Rightarrow x+y=0\Rightarrow y=-x$ Eigenvector $v_2=\left(\begin{array}{c}1\\ -1\end{array}\right)$.

Check: $A\left(\begin{array}{c}1\\ 1\end{array}\right)=\left(\begin{array}{c}3\\ 3\end{array}\right)=3\left(\begin{array}{c}1\\ 1\end{array}\right).$ ✓

Worked Example 2 — ek triangular shortcut

$B=\left(\begin{array}{cc}4& 2\\ 0& 5\end{array}\right)$

Step 1: $\det (B-\lambda I)=(4-\lambda )(5-\lambda )-(2)(0)=(4-\lambda )(5-\lambda )$. Zero term kyun? Lower-left $0$ hai, toh cross product vanish ho jaata hai.

Step 2: Roots hain $\lambda =4,5$ — exactly diagonal entries.

Do invariants jo mistakes jaldi pakad lete hain

Kyun (sketch): Characteristic polynomial $\det (A-\lambda I)=(-1{)}^n(\lambda -{\lambda }_1)\cdots (\lambda -{\lambda }_n)$. ${\lambda }^{n-1}$ coefficient match karne se sum = trace milta hai; constant term ($\lambda =0$) se $\det (A)=\prod {\lambda }_i$ milta hai.

Example 1 check: trace $=2+2=4=1+3$ ✓, $\det =4-1=3=1\times 3$ ✓.

Pehle Forecast, Phir Verify

Common Mistakes (Steel-manned)

Flashcards

Connections

• Determinants — woh tool jo singularity detect karta hai ($\det =0$).
• Matrix as Linear Transformation — eigenvectors = woh directions jo rotate nahi hote.
• Diagonalization — $A=P\Lambda P^{-1}$ eigenvectors ko $P$ ke columns ki tarah use karke.
• Principal Component Analysis (PCA) — covariance matrix ke eigenvectors.
• Symmetric Matrices and Spectral Theorem — real orthogonal eigenvectors.
• Trace and Determinant — invariants jo eigenvalues ke sum/product ke barabar hote hain.
• Null Space — eigenvectors $(A-\lambda I)$ ke null space mein rehte hain.

Concept Map