Symmetric matrices — spectral theorem (real eigenvalues, orthogonal eigenvectors)

WHAT is the theorem?

The decomposition $A = Q\Lambda Q^\top$ is called the spectral decomposition. Written out as a sum: $A = \sum_{i=1}^{n} \lambda_i\, \mathbf{q}_i \mathbf{q}_i^\top$ each $\mathbf{q}_i\mathbf{q}_i^\top$ is a rank-1 projection onto the $i$ -th eigen-axis, scaled by $\lambda_i$ .

Figure — Symmetric matrices — spectral theorem (real eigenvalues, orthogonal eigenvectors)

WHY are the eigenvalues real? (Derivation from scratch)

We do NOT assume eigenvalues are real — we prove it. Suppose $A\mathbf{x} = \lambda\mathbf{x}$ with possibly complex $\lambda$ and $\mathbf{x}\neq\mathbf{0}$ . Let $\bar{\mathbf{x}}$ be the complex conjugate and use the conjugate transpose $\mathbf{x}^* = \bar{\mathbf{x}}^\top$ .

Step 1 — Form the scalar $\mathbf{x}^* A \mathbf{x}$ . $\mathbf{x}^* A \mathbf{x} = \mathbf{x}^*(\lambda \mathbf{x}) = \lambda\,(\mathbf{x}^*\mathbf{x})$ Why this step? Plugging in $A\mathbf{x}=\lambda\mathbf{x}$ pulls $\lambda$ out front. Here $\mathbf{x}^*\mathbf{x} = \sum |x_i|^2 > 0$ is a positive real number.

Step 2 — Take the conjugate transpose of the same scalar. A $1\times1$ quantity equals its own conjugate transpose: $(\mathbf{x}^* A \mathbf{x})^* = \mathbf{x}^* A^* \mathbf{x}.$ Since $A$ is real and symmetric, $A^* = \bar{A}^\top = A^\top = A$ . So $(\mathbf{x}^* A \mathbf{x})^* = \mathbf{x}^* A \mathbf{x}.$ Why this step? Symmetry + realness is exactly what makes $\mathbf{x}^*A\mathbf{x}$ equal to its own conjugate — i.e. it is a real number.

Step 3 — Compare. From Step 1, $\mathbf{x}^*A\mathbf{x} = \lambda(\mathbf{x}^*\mathbf{x})$ , and its conjugate is $\bar\lambda(\mathbf{x}^*\mathbf{x})$ . Since the value is real: $\lambda(\mathbf{x}^*\mathbf{x}) = \bar\lambda(\mathbf{x}^*\mathbf{x}) \implies (\lambda - \bar\lambda)(\mathbf{x}^*\mathbf{x}) = 0.$ Because $\mathbf{x}^*\mathbf{x} > 0$ , we get $\lambda = \bar\lambda$ , so $\boxed{\lambda \in \mathbb{R}}$ . ∎

WHY are eigenvectors orthogonal? (Derivation from scratch)

Take two eigenpairs with distinct eigenvalues: $A\mathbf{u} = \lambda\mathbf{u}$ , $A\mathbf{v} = \mu\mathbf{v}$ , $\lambda \neq \mu$ .

Step 1 — Compute $\mathbf{u}^\top A \mathbf{v}$ two ways. $\mathbf{u}^\top A \mathbf{v} = \mathbf{u}^\top(\mu \mathbf{v}) = \mu\,(\mathbf{u}^\top \mathbf{v}).$ Why? Apply $A$ to $\mathbf{v}$ on the right.

Step 2 — Move $A$ to the left using symmetry. $\mathbf{u}^\top A \mathbf{v} = (A^\top \mathbf{u})^\top \mathbf{v} = (A\mathbf{u})^\top \mathbf{v} = (\lambda\mathbf{u})^\top\mathbf{v} = \lambda\,(\mathbf{u}^\top\mathbf{v}).$ Why? $\mathbf{u}^\top A = (A^\top\mathbf{u})^\top$ , and $A^\top = A$ lets us apply $A$ to $\mathbf{u}$ instead.

Step 3 — Equate. $\mu(\mathbf{u}^\top\mathbf{v}) = \lambda(\mathbf{u}^\top\mathbf{v}) \implies (\lambda - \mu)(\mathbf{u}^\top\mathbf{v}) = 0.$ Since $\lambda\neq\mu$ , we must have $\mathbf{u}^\top\mathbf{v} = 0$ , i.e. $\mathbf{u}\perp\mathbf{v}$ . ∎

WHY does $A = Q\Lambda Q^\top$ follow?

Stack the orthonormal eigenvectors as columns of $Q$ , so $Q^\top Q = I$ (i.e. $Q^{-1}=Q^\top$ ). The eigen-equations $A\mathbf{q}_i = \lambda_i\mathbf{q}_i$ packed together read: $AQ = Q\Lambda \quad\text{(column }i\text{ is }A\mathbf{q}_i = \lambda_i\mathbf{q}_i).$ Right-multiply by $Q^{-1} = Q^\top$ : $\boxed{A = Q\Lambda Q^\top.}$ Why this is special: a general diagonalizable matrix gives $A = P\Lambda P^{-1}$ with $P^{-1}\neq P^\top$ . Symmetry upgrades $P^{-1}$ to $Q^\top$ — that's the orthogonality bonus.

Worked Examples

Worked example Example 1 —

$2\times2$ , find $Q\Lambda Q^\top$ $A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$ .

Eigenvalues: $\det(A-\lambda I) = (2-\lambda)^2 - 1 = \lambda^2 - 4\lambda + 3 = (\lambda-1)(\lambda-3)$ . So $\lambda = 1, 3$ — both real ✓. Why expect real? $A$ is symmetric.

Eigenvectors:

$\lambda=3$ : $(A-3I)\mathbf{x}=\begin{pmatrix}-1&1\\1&-1\end{pmatrix}\mathbf{x}=0 \Rightarrow \mathbf{x}=(1,1)$ .
$\lambda=1$ : $(A-I)\mathbf{x}=\begin{pmatrix}1&1\\1&1\end{pmatrix}\mathbf{x}=0 \Rightarrow \mathbf{x}=(1,-1)$ .

Check orthogonality: $(1,1)\cdot(1,-1) = 1-1 = 0$ ✓. Why? Distinct eigenvalues ⇒ perpendicular.

Normalize: $\mathbf{q}_1 = \tfrac{1}{\sqrt2}(1,1)$ , $\mathbf{q}_2=\tfrac{1}{\sqrt2}(1,-1)$ . $Q = \frac{1}{\sqrt2}\begin{pmatrix}1&1\\1&-1\end{pmatrix},\quad \Lambda=\begin{pmatrix}3&0\\0&1\end{pmatrix},\quad A=Q\Lambda Q^\top.$

Worked example Example 2 — Spectral sum (rank-1 pieces)

Using Example 1, $A = 3\,\mathbf{q}_1\mathbf{q}_1^\top + 1\,\mathbf{q}_2\mathbf{q}_2^\top$ . $\mathbf{q}_1\mathbf{q}_1^\top = \tfrac12\begin{pmatrix}1&1\\1&1\end{pmatrix},\quad \mathbf{q}_2\mathbf{q}_2^\top=\tfrac12\begin{pmatrix}1&-1\\-1&1\end{pmatrix}.$ $3\cdot\tfrac12\begin{pmatrix}1&1\\1&1\end{pmatrix} + 1\cdot\tfrac12\begin{pmatrix}1&-1\\-1&1\end{pmatrix} = \begin{pmatrix}2&1\\1&2\end{pmatrix} = A\ ✓$ Why this step? It shows $A$ literally decomposes into orthogonal projectors scaled by eigenvalues.

Forecast-then-Verify

Common Mistakes (Steel-manned)

Flashcards

What two properties does the Spectral Theorem guarantee for a real symmetric matrix?

All eigenvalues are real, and eigenvectors of distinct eigenvalues are orthogonal (⇒ orthogonal diagonalization).

Write the orthogonal diagonalization form.

A = Q\Lambda Q^\top

with

Q^\top Q = I

and

\Lambda

diagonal of eigenvalues.

Why are the eigenvalues of a symmetric matrix real?

Because

\mathbf{x}^*A\mathbf{x}

equals its own conjugate (using

A^*=A

), forcing

\lambda=\bar\lambda

Key algebra step proving eigenvectors orthogonal?

\mu(\mathbf{u}^\top\mathbf{v})=\lambda(\mathbf{u}^\top\mathbf{v})

, so

(\lambda-\mu)(\mathbf{u}^\top\mathbf{v})=0

, hence

\mathbf{u}^\top\mathbf{v}=0

when

\lambda\neq\mu

Spectral sum form of

A

A=\sum_i \lambda_i\,\mathbf{q}_i\mathbf{q}_i^\top

(scaled orthogonal projectors).

Are symmetric matrices with repeated eigenvalues still diagonalizable?

Yes — eigenspaces always have full dimension; symmetric matrices are never defective.

What relation makes

Q

orthogonal?

Its columns are orthonormal eigenvectors, so

Q^\top = Q^{-1}

For a general (non-symmetric) matrix, are eigenvectors orthogonal?

No — orthogonality is a special property of symmetric/normal matrices.

Recall Feynman: explain to a 12-year-old

Imagine a stretchy rubber sheet. Most ways of pulling it would also twist and skew it. But a "symmetric" pull is special: it only stretches along certain straight lines — and those lines always cross each other at perfect right angles, like a clean grid. The amount of stretch along each line is the "eigenvalue," and the directions are the "eigenvectors." The Spectral Theorem just says: for these nice pulls, the stretch amounts are ordinary numbers (never weird imaginary ones), and the stretch directions are always neatly perpendicular. So you can always tilt your graph paper to line up with them and the whole thing becomes super simple.

Connections

Eigenvalues and Eigenvectors — the base concept this builds on.
Orthogonal Matrices — $Q$ here is exactly one of these ( $Q^\top Q=I$ ).
Diagonalization — symmetric case is the cleanest special case.
Quadratic Forms — $\mathbf{x}^\top A\mathbf{x}$ classified via signs of eigenvalues.
Positive Definite Matrices — symmetric with all $\lambda_i>0$ .
Singular Value Decomposition — SVD reduces to spectral theorem for symmetric $A$ .
Gram-Schmidt Process — used to orthonormalize repeated eigenspaces.

Concept Map

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho, symmetric matrix (jahan $A = A^\top$ , yaani diagonal ke aar-paar mirror image same hai) linear algebra ki sabse "sweet" matrix hai. Iska kaam sirf itna hai: space ko kuch perpendicular directions ke saath stretch karna — na koi ghumao (rotation), na koi tedha shear. Yeh perpendicular directions hi eigenvectors hain aur jitna stretch hota hai woh eigenvalues ( $\lambda$ ) hain. Spectral Theorem ka promise: symmetric matrix ke saare eigenvalues real honge (kabhi imaginary nahi), aur alag-alag eigenvalues ke eigenvectors hamesha 90 degree par honge.

Real eigenvalues kyun? Hum $\mathbf{x}^* A \mathbf{x}$ scalar banate hain. Symmetry ki wajah se yeh apne conjugate ke barabar ho jata hai, matlab woh ek

Symmetric matrices — spectral theorem (real eigenvalues, orthogonal eigenvectors)

WHAT is the theorem?

WHY are the eigenvalues real? (Derivation from scratch)

WHY are eigenvectors orthogonal? (Derivation from scratch)

WHY does $A = Q\Lambda Q^\top$ follow?

Worked Examples

Forecast-then-Verify

Common Mistakes (Steel-manned)

Flashcards

Connections

Concept Map

Hinglish (regional understanding)

Go deeper — visual, from zero

Test yourself — Linear Algebra (Full)

Connections