4.5.42 · D3Linear Algebra (Full)

Worked examples — Pseudoinverse

3,636 words17 min readBack to topic

This page is the drill hall for the Pseudoinverse. The parent note built the three formulas; here we throw every kind of matrix at them — tall, fat, square-invertible, square-singular, rank-deficient (both tall and fat flavours), and a real-world data-fit — so no scenario can surprise you in an exam.

Before we start, one promise: every symbol below was earned in the parent note. As a lightning refresher of the only three tools — plus the four rules that define what a pseudoinverse even is:

Recall The three formulas (from the parent)
  • Tall & full column rank ( independent columns, ): . This is a left inverse: .
  • Fat & full row rank ( independent rows, ): . This is a right inverse: .
  • Any shape, any rank: from the Singular Value Decomposition, inverting each nonzero , leaving zeros dead.

Here means "flip across its diagonal" (rows become columns), and means the ordinary length .

Recall The four Penrose conditions (what "

" means) — and the two parent mistakes we lean on is the unique matrix satisfying all four:

  1.    2.    3.    4. . Condition 1 is the one we use again and again below as a "plug-back" check.

Two traps from the parent note we will trip on purpose:

  • Mistake 1: " always." It needs full column rank; otherwise is singular (Examples 5, 8).
  • Mistake 2: " always." Only for full column rank. In general is a projection (it can have eigenvalue 0), seen in Examples 5, 8, 9.

The scenario matrix

Every matrix you can feed a pseudoinverse falls into exactly one of these boxes. The whole point of is that it never crashes — but which formula is safe changes box to box. Below, "rank" = number of genuinely independent rows/columns (directions the matrix does not squash to zero).

Cell Shape & rank What goes wrong with ? Which formula is safe Example
C-tall tall, full column rank not square — no ; has no exact solution (least squares) Ex 1, Ex 2
C-fat fat, full row rank not square; infinitely many solutions (min-norm) Ex 3
C-sqinv square, invertible nothing! exists any — all agree, Ex 4
C-sqsing square, singular (rank deficient) , no ; both and singular must use SVD Ex 5
C-zero the zero matrix (degenerate limit) everything is nullspace SVD gives (see Ex 6 for why) Ex 6
C-word real data fit (tall in disguise) overdetermined, noisy least squares Ex 7
C-twist-t exam trap: rank-deficient tall looks like C-tall but is singular! SVD, not the tall formula Ex 8
C-twist-f exam trap: rank-deficient fat looks like C-fat but is singular! SVD, not the fat formula Ex 9

The two figures below give the geometry behind the two "no exact inverse" cells: the tall case projects down (Ex 1), the fat case picks the shortest arrow (Ex 3).

Figure — Pseudoinverse
Figure — Pseudoinverse

Cell C-tall — least squares


Cell C-fat — minimum norm


Cell C-sqinv — the honest inverse


Cell C-sqsing — square but singular


Cell C-zero — the degenerate limit


Cell C-word — a real-world fit


Cell C-twist-t — the tall exam trap


Cell C-twist-f — the fat exam trap



Flashcards

Best single number fitting and (Ex 1)?
, the average; residual ⟂ column .
Min-norm solution of (Ex 3)?
, perpendicular to nullspace direction .
Why does a tall matrix sometimes still need SVD (Ex 8)?
If its columns are dependent (rank deficient), is singular so the tall formula fails.
Why does a fat matrix sometimes still need SVD (Ex 9)?
If its rows are dependent (rank deficient), is singular so the fat formula fails.
Pseudoinverse of (Ex 5)?
; note is a projection, not .
Pseudoinverse of the zero matrix (Ex 6)?
The zero matrix itself.
Best-fit resistance from at (Ex 7)?
.
Handy identity linking transpose and pseudoinverse (Ex 9)?
.

Connections

Concept Map

independent columns

independent rows

det nonzero

dependent rows or cols

only safe tool

Check shape and rank first

Tall full column rank

Fat full row rank

Square invertible

Rank deficient any shape

A+ = inv AtA At

A+ = At inv AAt

A+ = A inverse

A+ = V Sigma+ Ut