Foundations — Eigenvalue computation — power method, inverse iteration

Before you can read the parent note, you need to own every symbol it throws at you. We build them one at a time, each on top of the last. Nothing is assumed.

1. A vector — an arrow with a list of numbers

The stacked notation $\left(\genfrac{}{}{0ex}{}{x}{y}\right)$ is just a column of numbers. Reading top-to-bottom: first number = horizontal step, second = vertical step.

Why the topic needs it. Every method — power method, inverse iteration — is a story about what happens to an arrow when you feed it to a matrix again and again. No arrow, no story.

Why we care: the parent note normalizes every step ($x_{k+1}=y_{k+1}/\Vert y_{k+1}\Vert$). Dividing an arrow by its own length keeps its direction but resets its length to exactly $1$ — otherwise repeated stretching would blow the numbers up to infinity.

2. A matrix — the stretching machine

For a $2\times 2$ matrix the rule is: $\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)\left(\genfrac{}{}{0ex}{}{x}{y}\right)=\left(\genfrac{}{}{0ex}{}{ax+by}{cx+dy}\right).$

How to read this: the top output number comes from sliding the top row $(a,b)$ across the arrow $(x,y)$ and multiplying-then-adding pairs: $a\cdot x+b\cdot y$. Same for the bottom row.

Why the topic needs it. $A$ is the object whose eigenvalues we want. Everything reduces to "apply $A$" or "undo $A$."

3. Eigen-pairs — the arrows a matrix does NOT turn

Most arrows get rotated when you feed them to $A$. But a few special arrows come out pointing the same way, only longer or shorter. Those are the stars of the show.

The Greek letter $\lambda$ ("lambda") is just a name for the stretch factor. The equation $Av=\lambda v$ says: the machine's action on $v$ is identical to simply multiplying by the number $\lambda$.

Reading the signs of $\lambda$ — all cases:

• $\lambda >1$: same direction, longer.
• $\lambda =1$: unchanged (fixed arrow).
• $0<\lambda <1$: same direction, shorter.
• $\lambda =0$: collapses to the origin (matrix squashes this arrow flat).
• $\lambda <0$: flips to the opposite direction, then scaled by $|\lambda |$.

The $|\cdot |$ bars mean absolute value — the size of the number, ignoring its sign. The parent note ranks eigenvalues by $|\lambda |$ because it cares about how much stretch, not which way it points.

Why the topic needs it. The core promise — "repeated multiplication amplifies the most-stretched direction" — only makes sense once you know which direction stretches most. That is the dominant eigenvector $v_1$.

4. A basis — writing any arrow as a mix of eigenvectors

So any seed arrow splits into eigen-ingredients: $x_0=c_1{v}_1+c_2{v}_2+\cdots +c_n{v}_n.$ The numbers $c_i$ are the amounts (coefficients) of each eigenvector inside $x_0$. Think of a smoothie: $x_0$ is the drink, $v_i$ are the fruits, $c_i$ are how much of each you poured in.

Why the topic needs it. The whole power-method derivation lives in this coordinate system. Because $A$ acts simply on each $v_i$ (just multiply by ${\lambda }_i$), splitting $x_0$ into eigen-parts makes $A^k{x}_0$ easy to compute: $A^k{x}_0=c_1{\lambda }_1^k{v}_1+\cdots +c_n{\lambda }_n^k{v}_n.$ Each ingredient just gets its own stretch factor raised to the power $k$.

5. The exponent $k$ and the ratio ${\left(\frac{{\lambda }_2}{{\lambda }_1}\right)}^k$

The magic lever is the ratio $\frac{{\lambda }_2}{{\lambda }_1}$. Since $|{\lambda }_2|<|{\lambda }_1|$, this fraction has absolute value less than 1. Any number smaller than 1 in size, raised to a growing power $k$, shrinks toward $0$:

• ratio $=0.5\Rightarrow 0.5,0.25,0.125,\cdots \to 0$ fast.
• ratio $=0.95\Rightarrow$ shrinks, but slowly (convergence crawls).
• ratio $\to 1\Rightarrow$ barely shrinks — the two biggest eigenvalues are nearly tied and the method stalls.

Why the topic needs it. This shrinking is exactly why every non-dominant ingredient dies out, leaving only $v_1$. The size of this ratio is the convergence speed — see Convergence Rates.

6. The inverse $A^{-1}$ and solving $Ay=x$

Here $I$ is the identity matrix — 1's down the diagonal, 0's elsewhere — the "do-nothing" machine ($Ix=x$).

Key fact the parent uses: if $Av=\lambda v$ then $A^{-1}v=\frac{1}{\lambda }v$. Same eigenvector, reciprocal eigenvalue. Undoing a $\times 3$ stretch means a $\times \frac{1}{3}$ shrink — same direction.

The $2\times 2$ inverse formula (used in the parent's worked example): ${\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)}^{-1}=\frac{1}{ad-bc}\left(\begin{array}{cc}d& -b\\ -c& a\end{array}\right).$ The number $ad-bc$ is the determinant $\det$ — a single number measuring how much the matrix scales area. If $\det =0$ the machine flattens area to nothing and cannot be undone ($A^{-1}$ doesn't exist).

7. The shift $A-\sigma I$

Picture sliding a ruler: if the eigenvalues sit at marks $\{1.4,3.6\}$ and you shift by $\sigma =1.3$, they move to $\{0.1,2.3\}$. The one you targeted is now tiny — and after inverting, tiny becomes enormous, so it dominates. That is Rayleigh Quotient Iteration's engine and the parent's "hunt any eigenvalue" trick.

8. The Rayleigh quotient $\frac{x^{\top }Ax}{x^{\top }x}$

Two new pieces here: the dot product and the transpose $\top$.

So $x^{\top }x=x_1^2+x_2^2=\Vert x{\Vert }^2$: an arrow dotted with itself gives its length-squared.

Why the topic needs it. It reads the eigenvalue number off whatever arrow you currently hold — the "$\lambda$-meter." See Rayleigh Quotient Iteration.

The prerequisite map

This all feeds the parent: the topic note. Deeper machinery lives in Characteristic Polynomial, Spectral Decomposition, and QR Algorithm.

Equipment checklist

Test yourself — cover the right side.