4.8.21 · D5Numerical Methods

Question bank — Eigenvalue computation — power method, inverse iteration

1,355 words6 min readBack to topic

True or false — justify

The power method always converges to the eigenvector with the algebraically largest eigenvalue
False — it converges to the largest in magnitude . A matrix with eigenvalues and has dominant , not .
If the two largest eigenvalues satisfy , the power method still converges to a single eigenvector
False — the ratio never decays, so those two components persist and the iterate oscillates or rotates within the plane instead of settling.
The power method needs to be symmetric to work
False — it only needs a dominant eigenvalue and a full eigenbasis. Symmetry is a bonus: it makes the Rayleigh quotient accurate to rather than .
Inverse iteration and power method converge at the same rate on the same matrix
False — power method's rate is while inverse iteration's is ; these depend on different eigenvalue gaps and are usually very different.
Choosing the shift exactly equal to an eigenvalue makes shifted inverse iteration converge in zero steps
False and dangerous — then is exactly singular, so the system has no unique solution. In practice near-singularity is fine (even helpful); exact singularity breaks the solve.
Normalizing every step changes which eigenvector the method finds
False — normalization only rescales length; direction is untouched. It exists purely to stop the vector from overflowing () or underflowing.
The Rayleigh quotient can be computed only after the iteration has fully converged
False — it is a best-fit eigenvalue for the current vector, so it gives a usable (improving) estimate at every step, well before convergence.
Applying has the same eigenvectors as
True — from , multiply by to get ; same , reciprocal eigenvalue. See Spectral Decomposition.
Shifting by changes the eigenvectors of
False — keeps the same ; only eigenvalues slide by . That invariance is exactly why shift-and-invert can target a chosen eigenvector.

Spot the error

"To do inverse iteration, compute once, then power-iterate ."
The result is correct but the method is wrong: explicitly inverting is , numerically unstable, and wasteful. Instead solve each step, reusing one LU Decomposition — "solve, don't invert."
"Convergence of the power method is quadratic because grows exponentially."
growing fast is not convergence of direction. The error in direction decays like — a constant ratio per step, i.e. linear convergence, not quadratic.
"Since has for our matrix, we're guaranteed for any seed."
is the -component of the specific seed; a different seed lying in has . You must check each seed (random seeds work almost surely).
"The Rayleigh quotient rate for shifted inverse iteration is , so shrink it to zero."
The rate is the ratio where is the second-closest eigenvalue to . Only the ratio (numerator vs. distance to the next eigenvalue) controls speed, not the raw distance.
"The characteristic polynomial is unsolvable for , so eigenvalues literally cannot be found."
There is no closed-form radical formula for (Characteristic Polynomial), but eigenvalues are still computable to any precision by iterative methods like these or the QR Algorithm.
"After power method finds , run it again on the same to get ."
Plain power method keeps returning — the dominant direction re-dominates. To reach you must deflate (remove the component) or use a shift/inverse strategy.
"Rayleigh quotient — no denominator needed."
Only valid when is already normalized (). In general ; dropping the denominator on an unnormalized vector rescales your eigenvalue estimate.

Why questions

Why do we factor out in the derivation rather than ?
Factoring the largest term leaves ratios that are all and vanish; factoring anything smaller would leave a term that blows up, hiding the convergence.
Why does inverse iteration turn the smallest eigenvalue into the target?
's eigenvalues are , so 's smallest becomes the largest — the one power method naturally amplifies. See Convergence Rates.
Why does a shift near give explosive convergence?
becomes tiny, so becomes enormous and dwarfs every other reciprocal — the ratio to the next term collapses toward , so one or two steps can suffice.
Why use the Rayleigh quotient instead of just reading one component of ?
The Rayleigh quotient is the least-squares best eigenvalue over all directions of the current vector, so it averages out the not-yet-decayed error components; a single component is corrupted by whichever non-dominant piece happens to sit there.
Why LU-factor once instead of re-solving from scratch each step?
The matrix is the same at every iteration; only the right-hand side changes. One factorization plus cheap forward/back-substitutions per step beats repeated full solves.
Why does the eigenvalue gap being close to make the power method slow?
The non-dominant error decays as ; if that ratio , each step removes only of the error, so you need hundreds of iterations to converge.

Edge cases

What happens if is exactly an eigenvector (not )?
In exact arithmetic forever — you converge to the wrong eigenvector. In floating point, rounding usually re-injects a tiny component that eventually dominates.
What if has a zero eigenvalue and you try plain inverse iteration?
is singular, so (or the solve ) does not exist — inverse iteration cannot start. This is fine for shifted inverse iteration as long as is nonsingular.
What if the dominant eigenvalue is a complex conjugate pair (real )?
Then with the two complex conjugates tied in magnitude, so the real power method cannot pick a single real eigenvector and instead oscillates — a signal to switch to the QR Algorithm.
What if sits exactly halfway between two eigenvalues?
The two nearest eigenvalues are equidistant, so and the convergence ratio is — shifted inverse iteration stalls; nudge toward the one you actually want.
What happens as for a symmetric with distinct dominant eigenvalue — how accurate is ?
For symmetric the Rayleigh quotient error is , so is far more accurate than the vector itself and converges roughly twice as fast in exponent.
What if but the seed's is negative?
The iterate still aligns with the direction (up to sign); is just as valid an eigenvector as , and the Rayleigh quotient returns the same regardless of sign.

Recall One-line summary of the traps

Magnitude not sign; ratios not raw gaps; solve not invert; linear not quadratic; check ; and ties in magnitude () are where every method breaks.