Symmetric matrices — spectral theorem (real eigenvalues, orthogonal eigenvectors): Exercises

Reminders in one place, so you never meet a symbol cold:

A matrix is symmetric if $A = A^{⊤}$ — flipping it across its main diagonal (top-left to bottom-right) changes nothing. See the mirror picture below.
$A^{⊤}$ ("A transpose") means "swap rows with columns": the entry in row $i$ , column $j$ moves to row $j$ , column $i$ .
An eigenvector $x$ of $A$ is a non-zero arrow that $A$ only stretches, not turns: $A x = λ x$ . The stretch factor $λ$ ("lambda") is the eigenvalue. See Eigenvalues and Eigenvectors.
Two arrows are orthogonal (perpendicular) when their dot product is zero: $u \cdot v = u_{1} v_{1} + u_{2} v_{2} + \dots = 0$ .
$Q$ is orthogonal when its columns are perpendicular unit arrows, which forces $Q^{⊤} Q = I$ and hence $Q^{- 1} = Q^{⊤}$ . See Orthogonal Matrices.

Figure below — read it now, it recurs throughout. The left panel shows what "symmetric" looks like: the amber pair straddling the dashed mirror line are forced equal, which is precisely the condition Exercise 1.1 tests. The right panel shows the payoff proven in Exercise 3.1: the two eigen-axes meet at a clean $9 0^{\circ}$ and $A$ merely stretches along them (cyan by $3$ , amber by $1$ ) — no twisting. Keep this picture in mind; L2, L3 and L5 all live inside it.

Figure — Symmetric matrices — spectral theorem (real eigenvalues, orthogonal eigenvectors)

Level 1 — Recognition

Exercise 1.1

Which of these matrices are symmetric? $A = (1225), B = (0 - 3 30), C = 410147074 .$

Recall Solution

What we do: compare each matrix with its own transpose — mirror across the diagonal (exactly the dashed line in the figure above).

$A$ : the off-diagonal entries are $2$ (row 1) and $2$ (row 1 of transpose) — they match. Symmetric ✓.
$B$ : entry $(1, 2) = 3$ but entry $(2, 1) = - 3$ . Mirroring flips the sign, so $B^{⊤} = - B$ . Not symmetric (it is antisymmetric).
$C$ : check each mirror pair: $(1, 2) = 1 = (2, 1)$ , $(1, 3) = 0 = (3, 1)$ , $(2, 3) = 7 = (3, 2)$ . All match. Symmetric ✓.

Answer: $A$ and $C$ are symmetric; $B$ is not.

Exercise 1.2

For $A = (2112)$ , verify that $x = (1, 1)$ is an eigenvector and find its eigenvalue.

Recall Solution

What we do: multiply $A x$ and see whether the output is a scalar multiple of the input arrow. $A x = (2112) (11) = (2 \cdot 1 + 1 \cdot 1 1 \cdot 1 + 2 \cdot 1) = (33) = 3 (11) .$ The output points the same way as $(1, 1)$ , just $3 \times$ longer. So $x = (1, 1)$ is an eigenvector with eigenvalue $λ = 3$ .

Level 2 — Application

Exercise 2.1

Fully orthogonally diagonalize $A = (3113)$ : find $Q$ , $Λ$ with $A = Q Λ Q^{⊤}$ .

Recall Solution

Step 1 — eigenvalues. Why: eigenvalues are the stretch factors; find where $A - λ I$ collapses an arrow to zero, i.e. where $det (A - λ I) = 0$ . $det (3 - λ 1 1 3 - λ) = (3 - λ)^{2} - 1 = λ^{2} - 6 λ + 8 = (λ - 4) (λ - 2) .$ So $λ = 4$ and $λ = 2$ — both real (as the Spectral Theorem promises).

Step 2 — eigenvectors. Solve $(A - λ I) x = 0$ .

$λ = 4$ : $(- 1 1 1 - 1) x = 0 \Rightarrow - x_{1} + x_{2} = 0 \Rightarrow x = (1, 1)$ .
$λ = 2$ : $(1111) x = 0 \Rightarrow x_{1} + x_{2} = 0 \Rightarrow x = (1, - 1)$ .

Step 3 — orthogonality check. $(1, 1) \cdot (1, - 1) = 1 - 1 = 0$ ✓ (distinct eigenvalues ⇒ perpendicular — the right panel of the opening figure, with these exact axes).

Step 4 — normalize (divide by length $1^{2} + 1^{2} = 2$ so columns are unit arrows, making $Q$ orthogonal): $Q = \frac{1}{2} (11 1 - 1), Λ = (4002) .$ Then $A = Q Λ Q^{⊤}$ . See Diagonalization.

Exercise 2.2

Using Exercise 2.1's data, write $A$ as a spectral sum $λ_{1} q_{1} q_{1}^{⊤} + λ_{2} q_{2} q_{2}^{⊤}$ and confirm it reproduces $A$ .

Recall Solution

What we do: each $q_{i} q_{i}^{⊤}$ is a rank-1 projector onto eigen-axis $i$ ; scale by $λ_{i}$ and add. With $q_{1} = \frac{1}{2} (1, 1)$ , $q_{2} = \frac{1}{2} (1, - 1)$ : $q_{1} q_{1}^{⊤} = \frac{1}{2} (1111), q_{2} q_{2}^{⊤} = \frac{1}{2} (1 - 1 - 1 1) .$ $4 \cdot \frac{1}{2} (1111) + 2 \cdot \frac{1}{2} (1 - 1 - 1 1) = (2222) + (1 - 1 - 1 1) = (3113) = A ✓$

Level 3 — Analysis

Exercise 3.1

Show from scratch that if $A$ is symmetric and $u, v$ are eigenvectors with different eigenvalues $λ \neq = μ$ , then $u \cdot v = 0$ . (No numbers — a proof.)

Recall Solution

Idea: compute the single number $u^{⊤} A v$ in two different ways and compare. This is a classic "evaluate one quantity two ways" trick.

Way 1 — let $A$ act on $v$ : $u^{⊤} A v = u^{⊤} (μ v) = μ (u^{⊤} v) .$

Way 2 — slide $A$ onto $u$ using symmetry. Since $u^{⊤} A = (A^{⊤} u)^{⊤}$ and $A^{⊤} = A$ : $u^{⊤} A v = (A u)^{⊤} v = (λ u)^{⊤} v = λ (u^{⊤} v) .$

Compare: $μ (u^{⊤} v) = λ (u^{⊤} v) \Rightarrow (λ - μ) (u^{⊤} v) = 0$ . Since $λ \neq = μ$ the first factor is non-zero, so $u^{⊤} v = 0$ , i.e. $u ⊥ v$ . ∎ (Here $u^{⊤} v$ is just the dot product written as a matrix product.)

Exercise 3.2

The matrix $M = (1012)$ is not symmetric. Its eigenvectors are $(1, 0)$ (for $λ = 1$ ) and $(1, 1)$ (for $λ = 2$ ). Are they orthogonal? What does this reveal?

Recall Solution

Check: $(1, 0) \cdot (1, 1) = 1 \cdot 1 + 0 \cdot 1 = 1 \neq = 0$ . Not orthogonal. What it reveals: orthogonality of eigenvectors is a gift of symmetry, not a universal law. The proof in 3.1 used $A^{⊤} = A$ in "Way 2"; drop that hypothesis and the two ways no longer agree, so nothing forces $u \cdot v = 0$ . A non-symmetric matrix can still be diagonalized ( $M = P Λ P^{- 1}$ ), but $P^{- 1} \neq = P^{⊤}$ .

Level 4 — Synthesis

Exercise 4.1

A symmetric matrix with a repeated eigenvalue: $A = 500020002$ . Find all eigenvalues; then, treating the repeated eigenvalue carefully, produce a full orthonormal eigenbasis, running Gram–Schmidt inside the repeated eigenspace when the raw basis you pick is not already perpendicular.

Recall Solution

Step 1 — eigenvalues. $A$ is diagonal, so its eigenvalues are its diagonal entries: $λ = 5$ once, and $λ = 2$ with multiplicity 2. This is exactly the repeated-eigenvalue case.

Step 2 — the simple eigenvalue $λ = 5$ . Solving $(A - 5 I) x = 0$ forces $y = z = 0$ , giving the eigenvector $(1, 0, 0)$ — already unit length.

Step 3 — the repeated eigenvalue $λ = 2$ (the interesting case). Solving $(A - 2 I) x = 0$ : $300000000 x = 0 \Rightarrow x = 0, y, z free .$ The eigenspace is the whole $y$ – $z$ plane — a 2-dimensional space (symmetric matrices are never defective, so the eigenspace has full dimension). Suppose the two raw basis vectors we happen to pick are not already orthogonal, say $w_{1} = (0, 1, 0), w_{2} = (0, 1, 1) (w_{1} \cdot w_{2} = 1 \neq = 0) .$ Gram–Schmidt fixes this inside the eigenspace (Gram-Schmidt Process): $e_{1} = \frac{w _{1}}{∥ w _{1} ∥} = (0, 1, 0) .$ Subtract from $w_{2}$ its shadow along $e_{1}$ : $w_{2}^{⊥} = w_{2} - (w_{2} \cdot e_{1}) e_{1} = (0, 1, 1) - (1) (0, 1, 0) = (0, 0, 1),$ then normalize (already unit): $e_{2} = (0, 0, 1)$ . Now $e_{1} \cdot e_{2} = 0$ ✓.

Step 4 — assemble the full orthonormal eigenbasis. Combining the $λ = 5$ vector with the two orthonormalized $λ = 2$ vectors: ${(1, 0, 0), (0, 1, 0), (0, 0, 1)} .$ $Q = 100010001, Λ = 500020002, A = Q Λ Q^{⊤} .$ Moral: a distinct eigenvalue hands you its eigen-axis for free; a repeated eigenvalue gives you a whole plane, and inside it you must build an orthonormal basis by hand with Gram–Schmidt.

Exercise 4.2

Let $A$ be symmetric with spectral decomposition $A = \sum_{i} λ_{i} q_{i} q_{i}^{⊤}$ . Prove that $A^{2} = \sum_{i} λ_{i}^{2} q_{i} q_{i}^{⊤}$ , and use it to state $A^{k}$ .

Recall Solution

Key fact: the $q_{i}$ are orthonormal, so $q_{i}^{⊤} q_{j} = 1$ if $i = j$ and $0$ otherwise. Square via $A = Q Λ Q^{⊤}$ : $A^{2} = (Q Λ Q^{⊤}) (Q Λ Q^{⊤}) = Q Λ (Q^{⊤} Q) Λ Q^{⊤} = Q Λ I Λ Q^{⊤} = Q Λ^{2} Q^{⊤} .$ The middle $Q^{⊤} Q = I$ collapses (that is orthogonality doing the work). Since $Λ^{2}$ is diagonal with entries $λ_{i}^{2}$ : $A^{2} = \sum_{i} λ_{i}^{2} q_{i} q_{i}^{⊤} .$ General power: $A^{k} = Q Λ^{k} Q^{⊤} = \sum_{i} λ_{i}^{k} q_{i} q_{i}^{⊤}$ . The eigen-axes are frozen; only the stretch factors get raised to the power.

Level 5 — Mastery

Exercise 5.1

For $A = (3113)$ (from Exercise 2.1, eigenvalues $4, 2$ ), the quadratic form $q (x) = x^{⊤} A x$ traces a curve when set to $q (x) = 1$ . Using the eigen-axes, identify the shape, its axis directions, and its semi-axis lengths.

Recall Solution

Rotate into eigen-coordinates. Put $x = Q y$ with $Q$ from Exercise 2.1. Then $q = x^{⊤} A x = (Q y)^{⊤} (Q Λ Q^{⊤}) (Q y) = y^{⊤} Λ y = 4 y_{1}^{2} + 2 y_{2}^{2} .$ The cross term $2 x_{1} x_{2}$ vanished — that is exactly what the eigen-axes buy you. See Quadratic Forms. Set $= 1$ : $4 y_{1}^{2} + 2 y_{2}^{2} = 1$ , i.e. $\frac{y _{1}^{2}}{( 1/2 ) ^{2}} + \frac{y _{2}^{2}}{( 1/ 2 ) ^{2}} = 1$ . This is an ellipse (drawn in the figure below).

Semi-axis along $q_{1} = \frac{1}{2} (1, 1)$ has length $1/ λ_{1} = 1/ 4 = \frac{1}{2}$ .
Semi-axis along $q_{2} = \frac{1}{2} (1, - 1)$ has length $1/ λ_{2} = 1/ 2 \approx 0.707$ .

Axis directions in the original $x$ –coordinates: the ellipse's short axis points along $q_{1} = \frac{1}{2} (1, 1)$ (the $4 5^{\circ}$ line), and its long axis points along $q_{2} = \frac{1}{2} (1, - 1)$ (the $- 4 5^{\circ}$ line). These are exactly the eigen-axes — the ellipse is tilted at $4 5^{\circ}$ , not aligned with the $x$ – and $y$ –axes.

Reading it off: larger eigenvalue ⇒ steeper form ⇒ shorter semi-axis, so $λ_{1} = 4$ gives the short axis $\frac{1}{2}$ and $λ_{2} = 2$ gives the long axis $1/ 2$ . Both eigenvalues positive ⇒ the curve closes into an ellipse ⇒ $A$ is positive definite.

Exercise 5.2

Let $A = (2112)$ . Its singular values are the numbers appearing in the SVD. Explain why, for this $A$ , the singular values equal $∣ λ_{1} ∣, ∣ λ_{2} ∣$ , and give them numerically.

Recall Solution

Eigenvalues (parent Example 1): $λ = 3, 1$ . Singular values are $eigenvalues of A^{⊤} A$ . For symmetric $A$ , $A^{⊤} A = A^{2}$ , whose eigenvalues are $λ_{i}^{2}$ . So singular values $= λ_{i}^{2} = ∣ λ_{i} ∣$ . Here $∣3∣ = 3$ , $∣1∣ = 1$ , so singular values are $3 and 1$ . Subtlety: they equal $∣ λ_{i} ∣$ , the absolute values. If an eigenvalue were negative, its singular value would be its magnitude — the sign gets absorbed into the SVD's orthogonal factors.

Exercise 5.3

Suppose symmetric $A$ has eigenvalues $λ = 4$ (eigenvector $\frac{1}{2} (1, 1)$ ) and $λ = - 2$ (eigenvector $\frac{1}{2} (1, - 1)$ ). Reconstruct $A$ from its spectral sum.

Recall Solution

$A = 4 \cdot \frac{1}{2} (1111) + (- 2) \cdot \frac{1}{2} (1 - 1 - 1 1) = (2222) + (- 1 1 1 - 1) = (1331) .$ Check: trace $= 2 = 4 + (- 2)$ ✓, $det = 1 - 9 = - 8 = 4 \cdot (- 2)$ ✓. The negative eigenvalue makes $A$ indefinite — the quadratic form $q (x) = 1$ would be a hyperbola, not an ellipse.

Flashcards

For a real symmetric matrix, why must eigenvectors of distinct eigenvalues be orthogonal?

Computing

u^{⊤} A v

two ways gives

(λ - μ) (u^{⊤} v) = 0

; distinct

λ \neq = μ

forces

u^{⊤} v = 0

What are the singular values of a symmetric matrix in terms of its eigenvalues?

The absolute values

∣ λ_{i} ∣

For symmetric

A = Q Λ Q^{⊤}

, what is

A^{k}

A^{k} = Q Λ^{k} Q^{⊤} = \sum_{i} λ_{i}^{k} q_{i} q_{i}^{⊤}

— same axes, powered stretches.