Eigenvalues & Eigenvectors

The unifying principle

Let $A$ be a linear operator on a vector space. A nonzero vector $v$ is an eigenvector with eigenvalue $\lambda$ if

$A v = \lambda v.$

Geometrically: $A$ leaves the line through $v$ invariant, merely scaling it by $\lambda$ . Solving $\det(A - \lambda I) = 0$ gives the spectrum $\{\lambda_i\}$ .

The payoff is diagonalization. If $A$ has a full set of independent eigenvectors forming columns of $P$ , then

$A = P \Lambda P^{-1}, \qquad \Lambda = \mathrm{diag}(\lambda_1,\dots,\lambda_n).$

In the eigenbasis the operator is decoupled: every action of $A$ becomes coordinate-wise multiplication. For symmetric/Hermitian $A$ the eigenvectors are orthonormal ( $A = Q\Lambda Q^\top$ , the spectral theorem), and powers and exponentials become trivial:

$A^k = P\Lambda^k P^{-1}, \qquad e^{tA} = P\,e^{t\Lambda}\,P^{-1}.$

That last identity is why eigenvalues govern dynamics, stability, growth, and equilibrium everywhere.

How it shows up in each field

Maths — the spectral theorem and operator structure

The eigenvalues encode invariants: $\operatorname{tr}A = \sum_i \lambda_i$ , $\det A = \prod_i \lambda_i$ . The largest $|\lambda|$ controls long-term behavior of iteration $x_{k+1}=Ax_k$ .

Physics — normal modes & quantum observables

Two faces, same math.

Classical normal modes. Coupled oscillators obey $M\ddot{x} = -Kx$ . Seeking $x(t)=v\,e^{i\omega t}$ turns this into the generalized eigenproblem

$K v = \omega^2 M v.$

Eigenvectors $v$ are the normal modes (collective patterns of motion); eigenvalues $\omega^2$ are the squared frequencies. Any motion is a superposition of independently oscillating modes.

Quantum mechanics. Observables are Hermitian operators $\hat{H}$ . The stationary states solve

$\hat{H}\,\psi = E\,\psi,$

the time-independent Schrödinger equation — an eigenvalue problem. Eigenvalues $E$ are the only measurable energies; eigenvectors $\psi$ are the states that don't change shape under evolution ( $e^{-i\hat H t/\hbar}\psi = e^{-iEt/\hbar}\psi$ ). Hermiticity guarantees real eigenvalues and orthogonal states.

AI-ML — Principal Component Analysis & spectral methods

PCA seeks directions of maximal variance. For a centered data matrix $X$ , the covariance is $C=\frac1n X^\top X$ , and the principal directions solve

$C v_i = \lambda_i v_i.$

Coding/CS — PageRank & the dominant eigenvector

A web graph gives a column-stochastic transition matrix $M$ (with teleportation). PageRank is the steady-state of a random surfer:

$M\,r = r,$

i.e. the eigenvector of eigenvalue $\lambda=1$ . The Perron–Frobenius theorem guarantees a unique positive such vector for a primitive stochastic matrix, and that $\lambda=1$ is dominant.

Worked example Power iteration

r = np.ones(n) / n
for _ in range(100):
    r = M @ r          # repeatedly apply operator
    r /= r.sum()       # renormalize
# r → dominant eigenvector (PageRank scores)

The gap between the top two eigenvalues ( $|\lambda_2|$ , the "spectral gap") sets the convergence rate — and the same idea underlies Markov-chain mixing times and graph connectivity.

Why this bridge matters

What transfers is a single mental move: change basis to the eigenbasis and the problem decouples.

From physics → ML: A normal mode and a principal component are the same object — an orthogonal axis of a symmetric operator. Knowing modes "ring independently" gives instant intuition for why PCA components are uncorrelated.
From ML → CS: PCA's "dominant eigenvector = direction of most variance" mirrors PageRank's "dominant eigenvector = steady-state importance." Power iteration is repeated operator application converging to that axis.
From CS → physics: The spectral gap controlling Markov mixing is the same gap controlling how fast a quantum/classical system relaxes toward its ground/equilibrium mode.
Universal lever: Eigenvalues tell you stability (do iterates explode or decay?), frequency (how fast does a mode oscillate?), and importance/variance (how much "energy" lives on this axis?) — three readings of one number $\lambda$ .

The deepest transfer: Hermitian/symmetric operators always give a clean orthogonal eigenbasis. That single theorem licenses real measurement outcomes in QM, uncorrelated components in PCA, real frequencies for stable oscillators, and well-behaved spectral graph algorithms — all at once.

Connections

01 Linear Algebra — Diagonalization & Spectral Theorem
02 Differential Equations — Stability & $e^{tA}$
03 Classical Mechanics — Coupled Oscillators & Normal Modes
04 Quantum Mechanics — Schrödinger Eigenstates & Observables
05 Dimensionality Reduction — PCA & SVD
06 Spectral Methods — Graph Laplacian & Clustering
07 Markov Chains — Stationary Distributions & Mixing
08 PageRank — Power Iteration & Perron–Frobenius

#bridge