This page is the drill hall for the Pseudoinverse . The parent note built the three formulas; here we throw every kind of matrix at them — tall, fat, square-invertible, square-singular, rank-deficient (both tall and fat flavours), and a real-world data-fit — so no scenario can surprise you in an exam.
Before we start, one promise: every symbol below was earned in the parent note. As a lightning refresher of the only three tools — plus the four rules that define what a pseudoinverse even is:
Recall The three formulas (from the parent)
Tall & full column rank (n independent columns, m ≥ n ): A + = ( A ⊤ A ) − 1 A ⊤ . This is a left inverse: A + A = I n .
Fat & full row rank (m independent rows, n ≥ m ): A + = A ⊤ ( A A ⊤ ) − 1 . This is a right inverse: A A + = I m .
Any shape, any rank : A + = V Σ + U ⊤ from the Singular Value Decomposition , inverting each nonzero σ i , leaving zeros dead.
Here A ⊤ means "flip A across its diagonal" (rows become columns), and ∥ x ∥ means the ordinary length x 1 2 + x 2 2 + ⋯ .
Recall The four Penrose conditions (what "
A + " means) — and the two parent mistakes we lean on
A + is the unique matrix satisfying all four:
A A + A = A 2. A + A A + = A + 3. ( A A + ) ⊤ = A A + 4. ( A + A ) ⊤ = A + A .
Condition 1 is the one we use again and again below as a "plug-back" check.
Two traps from the parent note we will trip on purpose:
Mistake 1: "A + = ( A ⊤ A ) − 1 A ⊤ always ." It needs full column rank ; otherwise A ⊤ A is singular (Examples 5, 8).
Mistake 2: "A + A = I always ." Only for full column rank. In general A + A is a projection (it can have eigenvalue 0), seen in Examples 5, 8, 9.
Every matrix you can feed a pseudoinverse falls into exactly one of these boxes. The whole point of A + is that it never crashes — but which formula is safe changes box to box. Below, "rank" = number of genuinely independent rows/columns (directions the matrix does not squash to zero).
Cell
Shape & rank
What goes wrong with A − 1 ?
Which formula is safe
Example
C-tall
tall, full column rank
not square — no A − 1 ; A x = b has no exact solution
( A ⊤ A ) − 1 A ⊤ (least squares)
Ex 1, Ex 2
C-fat
fat, full row rank
not square; infinitely many solutions
A ⊤ ( A A ⊤ ) − 1 (min-norm)
Ex 3
C-sqinv
square, invertible
nothing! A − 1 exists
any — all agree, A + = A − 1
Ex 4
C-sqsing
square, singular (rank deficient)
det = 0 , no A − 1 ; both A ⊤ A and A A ⊤ singular
must use SVD
Ex 5
C-zero
the zero matrix (degenerate limit)
everything is nullspace
SVD gives A + = 0 (see Ex 6 for why)
Ex 6
C-word
real data fit (tall in disguise)
overdetermined, noisy
least squares
Ex 7
C-twist-t
exam trap: rank-deficient tall
looks like C-tall but A ⊤ A is singular!
SVD, not the tall formula
Ex 8
C-twist-f
exam trap: rank-deficient fat
looks like C-fat but A A ⊤ is singular!
SVD, not the fat formula
Ex 9
The two figures below give the geometry behind the two "no exact inverse" cells: the tall case projects b down (Ex 1), the fat case picks the shortest arrow (Ex 3).
Worked example Example 1 (cell
C-tall ) — the projection picture
A = ( 1 1 ) , b = ( 3 1 ) . Solve A x = b as best possible (x is a single number).
Forecast: guess the single best number x before reading. (The two equations x = 3 and x = 1 disagree — what's the fair compromise?)
Step 1 — spot the cell. A is 2 × 1 : two rows, one column. One column, and it's nonzero, so full column rank → cell C-tall → use A + = ( A ⊤ A ) − 1 A ⊤ .
Why this step? We must classify before we pick a formula; the tall formula is only legal when the column(s) are independent.
Step 2 — build A ⊤ A . A ⊤ A = ( 1 1 ) ( 1 1 ) = 2 .
Why this step? A ⊤ A is the tiny square matrix (here just the number 2) we must invert; it is invertible exactly because the column is independent.
Step 3 — assemble A + . A + = ( 2 ) − 1 ( 1 1 ) = ( 2 1 2 1 ) .
Why this step? This row vector is the refund machine: multiply any b by it and out comes the best x .
Step 4 — apply to b . x = A + b = 2 1 ⋅ 3 + 2 1 ⋅ 1 = 2 .
Why this step? x is literally the average of 3 and 1 — the fairest compromise, exactly as intuition demands.
What it looks like: in figure s01, b = ( 3 , 1 ) is dropped straight down onto the line spanned by ( 1 , 1 ) ; the foot of that perpendicular is A x = ( 2 , 2 ) , and the leftover arrow b − A x = ( 1 , − 1 ) is the residual , perpendicular to the line.
Verify: the residual b − A x = ( 3 , 1 ) − ( 2 , 2 ) = ( 1 , − 1 ) must be orthogonal to the column ( 1 , 1 ) : dot product 1 ⋅ 1 + ( − 1 ) ⋅ 1 = 0 ✓. Orthogonality of the residual is the least-squares condition (see Orthogonal Projection ).
Worked example Example 2 (cell
C-tall ) — the parent's line-fit, re-verified
Same as the parent's Example 1: A = 1 1 1 0 1 2 , b = 6 0 0 , fitting y = c + d t .
Forecast: three points ( 0 , 6 ) , ( 1 , 0 ) , ( 2 , 0 ) — no straight line hits all three. Guess the slope's sign.
Step 1 — cell. 3 × 2 , columns ( 1 , 1 , 1 ) and ( 0 , 1 , 2 ) are independent → C-tall.
Why this step? Same test as before; here we confirm rank 2 = number of columns.
Step 2 — A ⊤ A and A ⊤ b . A ⊤ A = ( 3 3 3 5 ) , A ⊤ b = ( 6 0 ) .
Why this step? These are the normal equations A ⊤ A x = A ⊤ b .
Step 3 — invert the 2 × 2 . det = 3 ⋅ 5 − 3 ⋅ 3 = 6 , so ( A ⊤ A ) − 1 = 6 1 ( 5 − 3 − 3 3 ) .
Why this step? A 2 × 2 inverse is d e t 1 ( d − c − b a ) .
Step 4 — solve. x = 6 1 ( 5 − 3 − 3 3 ) ( 6 0 ) = ( 5 − 3 ) . Best line y = 5 − 3 t . Slope is negative .
Why this step? Solving the normal equations yields the least-squares coefficients ( c , d ) = ( 5 , − 3 ) .
Verify: predictions ( 5 , 2 , − 1 ) at t = 0 , 1 , 2 ; residual r = b − A x = ( 6 , 0 , 0 ) − ( 5 , 2 , − 1 ) = ( 1 , − 2 , 1 ) . Check A ⊤ r = 0 : column ( 1 , 1 , 1 ) ⋅ ( 1 , − 2 , 1 ) = 0 ✓, column ( 0 , 1 , 2 ) ⋅ ( 1 , − 2 , 1 ) = 0 − 2 + 2 = 0 ✓.
Worked example Example 3 (cell
C-fat ) — the shortest arrow
A = ( 1 2 ) , b = ( 4 ) . Solve x 1 + 2 x 2 = 4 with the smallest ∥ x ∥ .
Forecast: the solutions form a whole line in the plane. Which point on that line is closest to the origin?
Step 1 — cell. 1 × 2 : one row, and it's nonzero → full row rank → cell C-fat → use A + = A ⊤ ( A A ⊤ ) − 1 .
Why this step? One equation, two unknowns: infinitely many solutions. We need the shortest one, which is the fat formula's job.
Step 2 — force x into the row space. The parent's derivation writes x = A ⊤ w , where w is a small helper vector (here a single unknown number) chosen so that the true equation A x = b is satisfied. Substituting x = A ⊤ w turns A x = b into A A ⊤ w = b .
Why this step? Every solution splits into a row-space part plus nullspace junk; writing x = A ⊤ w builds x purely out of the row direction ( 1 , 2 ) , guaranteeing zero junk — which is exactly what "smallest norm" demands. The letter w is just the coefficient telling us how much of the row direction to take.
Step 3 — solve for w , then x . A A ⊤ = ( 1 2 ) ( 1 2 ) = 1 + 4 = 5 , so w = ( A A ⊤ ) − 1 b = 5 1 ⋅ 4 = 5 4 . Then x = A ⊤ w = ( 1 2 ) ⋅ 5 4 = ( 4/5 8/5 ) .
Why this step? Once w is known, x = A ⊤ w hands us the answer directly. Bundling the two steps gives the closed form A + = A ⊤ ( A A ⊤ ) − 1 = ( 1/5 2/5 ) , and indeed A + b = ( 4/5 8/5 ) .
What it looks like: in figure s02 the dashed line is all solutions of x 1 + 2 x 2 = 4 ; the plum arrow to ( 4/5 , 8/5 ) is perpendicular to that line — the closest point to the origin.
Verify (two ways): (i) it solves the equation: 5 4 + 2 ⋅ 5 8 = 5 4 + 5 16 = 5 20 = 4 ✓. (ii) x is perpendicular to the nullspace direction ( 2 , − 1 ) : 5 4 ⋅ 2 + 5 8 ⋅ ( − 1 ) = 5 8 − 5 8 = 0 ✓ (see Four Fundamental Subspaces ).
Worked example Example 4 (cell
C-sqinv ) — A + collapses to A − 1
A = ( 2 1 1 1 ) . Since det A = 2 − 1 = 1 = 0 , A is square and invertible.
Forecast: the pseudoinverse should give exactly the ordinary inverse. Predict A + .
Step 1 — cell. Square, det = 0 → C-sqinv → all formulas agree and equal A − 1 .
Why this step? When A − 1 exists it uniquely satisfies all four Penrose conditions, so A + = A − 1 .
Step 2 — ordinary inverse. A − 1 = 1 1 ( 1 − 1 − 1 2 ) = ( 1 − 1 − 1 2 ) .
Why this step? The 2 × 2 inverse formula with det = 1 .
Step 3 — sanity via the tall formula. With full column rank the tall formula must give the same thing: A + = ( A ⊤ A ) − 1 A ⊤ . Since A ⊤ A is invertible and A square, ( A ⊤ A ) − 1 A ⊤ = A − 1 ( A ⊤ ) − 1 A ⊤ = A − 1 .
Why this step? This algebraic collapse is the whole reason A + "generalizes" the inverse (see Matrix Inverse ).
Verify: A A − 1 = ( 2 1 1 1 ) ( 1 − 1 − 1 2 ) = ( 1 0 0 1 ) = I ✓.
Worked example Example 5 (cell
C-sqsing ) — when det = 0 , only SVD survives
A = ( 1 1 1 1 ) . Here det = 1 − 1 = 0 : not invertible, and both A ⊤ A and A A ⊤ are singular, so neither the tall nor the fat formula is legal.
Forecast: A crushes everything onto the line ( 1 , 1 ) . The refund can only work along that one surviving direction. Guess how "spread out" A + looks.
Step 1 — cell. Square, det = 0 → C-sqsing → must use A + = V Σ + U ⊤ .
Why this step? The parent's Mistake 1 warned us: A ⊤ A singular ⇒ tall formula explodes.
Step 2 — find the surviving direction. A has rank 1: it scales the direction u = 2 1 ( 1 , 1 ) by 2 and kills the perpendicular direction u ⊥ = 2 1 ( 1 , − 1 ) . So the single nonzero singular value is σ 1 = 2 .
Why this step? A u = 2 1 ( 2 , 2 ) = 2 u and A u ⊥ = 2 1 ( 0 , 0 ) = 0 : these are the Eigenvalues and Eigenvectors 2 and 0 , which here coincide with the singular directions because A is symmetric.
Step 3 — write out the full U , Σ , V explicitly. Because A is symmetric and its eigenvalues 2 , 0 are non-negative, we may take U = V built column-by-column from u (for σ 1 = 2 ) then u ⊥ (for σ 2 = 0 ):
U = V = ( 2 1 2 1 2 1 − 2 1 ) , Σ = ( 2 0 0 0 ) .
Why this step? U and V must be genuine 2 × 2 orthonormal matrices; their columns are the unit singular vectors in singular-value order. You can check U ⊤ U = I (columns are perpendicular unit vectors) and U Σ V ⊤ = A .
Step 4 — invert nonzeros only. Σ + = ( 1/2 0 0 0 ) (flip 2 to 1/2, leave the 0 dead).
Why this step? Dividing by zero is impossible; the dead direction carries no information to restore.
Step 5 — reassemble. A + = V Σ + U ⊤ = 2 1 u u ⊤ = 2 1 ⋅ 2 1 ( 1 1 1 1 ) = ( 1/4 1/4 1/4 1/4 ) .
Why this step? Only the surviving direction u is rebuilt, scaled by 1/ σ 1 = 1/2 .
Verify (Penrose condition 1): A A + A = A . Compute A + A = ( 1/2 1/2 1/2 1/2 ) (a projection onto the row space, not I — parent's Mistake 2!). Then A ⋅ ( A + A ) = ( 1 1 1 1 ) = A ✓. Also confirm U Σ V ⊤ = A : U Σ = ( 2 2 0 0 ) , times V ⊤ gives ( 1 1 1 1 ) ✓.
Worked example Example 6 (cell
C-zero ) — the all-zero matrix
A = ( 0 0 0 0 ) . Every direction is nullspace. What is A + ?
Forecast: if A throws everything away, the fairest refund gives... what?
Step 1 — cell. Zero matrix → C-zero, the extreme of C-sqsing (rank 0) → SVD.
Why this step? No formula involving ( A ⊤ A ) − 1 can work; there is nothing to invert.
Step 2 — SVD. All singular values are 0 : Σ = 0 , hence Σ + = 0 (invert nothing , since no σ i is nonzero).
Why this step? The rule "invert nonzeros, leave zeros" leaves everything zero.
Step 3 — assemble. A + = V ⋅ 0 ⋅ U ⊤ = ( 0 0 0 0 ) .
Why this step? With no invertible directions, the best refund is "give nothing back" — which is exactly x = 0 , the minimum-norm least-squares answer.
Verify (why A + = 0 is the right answer, not a cop-out): the pseudoinverse must give the least-squares, minimum-norm solution of A x = b . For A = 0 , every x gives error ∥0 ⋅ x − b ∥ = ∥ b ∥ , so the error can't be reduced; among all x tied for that error, the smallest-norm one is x = 0 . Since x = A + b = 0 for every b , we need A + = 0 — the unique matrix delivering the fairest refund. All four Penrose conditions then hold trivially (A A + A = 0 = A ✓).
Worked example Example 7 (cell
C-word ) — resistance from noisy readings
A physicist measures voltage V across a resistor at currents I = 1 , 2 , 3 amps, getting V = 2.1 , 3.9 , 6.1 volts. Ohm's law says V = R I (a line through the origin). Find the best resistance R in ohms.
Forecast: the readings are near a slope-2 line. Guess R ≈ 2 Ω .
Step 1 — model as A x = b . A = 1 2 3 (currents), x = ( R ) , b = 2.1 3.9 6.1 (voltages).
Why this step? One unknown R , three noisy equations → overdetermined → cell C-word = C-tall in disguise.
Step 2 — cell & formula. 3 × 1 , nonzero column → full column rank → R = ( A ⊤ A ) − 1 A ⊤ b .
Why this step? Least squares over the noise, exactly like Example 1.
Step 3 — compute A ⊤ A and A ⊤ b . A ⊤ A = 1 2 + 2 2 + 3 2 = 1 + 4 + 9 = 14 . A ⊤ b = 1 ( 2.1 ) + 2 ( 3.9 ) + 3 ( 6.1 ) = 2.1 + 7.8 + 18.3 = 28.2 .
Why this step? These two numbers are all the normal equation A ⊤ A R = A ⊤ b needs.
Step 4 — solve for R . R = A ⊤ A A ⊤ b = 14 28.2 = 2.0142857 … ≈ 2.01 Ω .
Why this step? The formula weights higher-current readings more (larger I 2 ), which is correct: they carry more signal.
Verify (units + sanity): [ A ⊤ b ] = amp·volt, [ A ⊤ A ] = amp2 , ratio has units volt/amp = Ω ✓. Value ≈ 2.01 Ω matches the forecast of ≈ 2 Ω ✓ (application of Least Squares Regression ).
Worked example Example 8 (cell
C-twist-t ) — a tall matrix where the tall formula still fails
A = 1 2 3 2 4 6 . It's 3 × 2 (tall), so a hasty student reaches for ( A ⊤ A ) − 1 A ⊤ . Trap: column 2 is exactly 2 × column 1, so rank = 1 , and A ⊤ A is singular . Find A + correctly.
Forecast: will the tall formula work? (No — and this is the exam's whole point.)
Step 1 — detect the trap. Columns ( 1 , 2 , 3 ) and ( 2 , 4 , 6 ) are proportional → rank 1 < 2 → not full column rank → cell C-twist-t → SVD required .
Why this step? Tall shape is necessary but not sufficient (parent's Mistake 1); we need independent columns.
Step 2 — write A as one outer product (rank-1). A = 1 2 3 ( 1 2 ) , i.e. A = c r ⊤ with column-direction c = ( 1 , 2 , 3 ) and row-direction r = ( 1 , 2 ) .
Why this step? Every rank-1 matrix is a single column times a single row; this makes the SVD trivial to read off.
Step 3 — get the unit singular vectors and σ 1 . Normalise: the left singular vector is u = ∥ c ∥ c = 14 1 ( 1 , 2 , 3 ) , and the right singular vector is v = ∥ r ∥ r = 5 1 ( 1 , 2 ) . The lone singular value is σ 1 = ∥ c ∥ ∥ r ∥ = 14 ⋅ 5 = 70 .
Why this step? Since A = c r ⊤ = ( ∥ c ∥ u ) ( ∥ r ∥ v ) ⊤ = ( ∥ c ∥∥ r ∥ ) u v ⊤ , comparing to A = σ 1 u v ⊤ forces σ 1 = ∥ c ∥∥ r ∥ . Here u is now explicitly defined — the normalised column direction.
Step 4 — pseudoinverse of a rank-1 matrix. For A = σ 1 u v ⊤ , the SVD rule gives A + = σ 1 1 v u ⊤ . Substituting:
A + = 70 1 ⋅ 5 1 ( 1 2 ) ⋅ 14 1 ( 1 2 3 ) = 70 1 ( 1 2 2 4 3 6 ) .
Why this step? 70 ⋅ 5 ⋅ 14 = 70 ⋅ 70 = 70 ; only the surviving direction is inverted.
Verify (Penrose condition 1): A A + A = A . Since A + A = v v ⊤ = 5 1 ( 1 2 2 4 ) (row-space projection, not I — Mistake 2 again), A ( A + A ) : first column 5 1 [ ( 1 , 2 , 3 ) ⋅ 1 + ( 2 , 4 , 6 ) ⋅ 2 ] = 5 1 ( 5 , 10 , 15 ) = ( 1 , 2 , 3 ) ✓, recovering column 1 of A ; column 2 similarly gives ( 2 , 4 , 6 ) ✓.
Worked example Example 9 (cell
C-twist-f ) — a fat matrix where the fat formula still fails
A = ( 1 2 2 4 3 6 ) . It's 2 × 3 (fat), so a hasty student reaches for A ⊤ ( A A ⊤ ) − 1 . Trap: row 2 is exactly 2 × row 1, so rank = 1 , and A A ⊤ is singular — the direct analogue of Example 8's trap, one dimension over.
Forecast: will the fat formula work? (No — and it's the mirror image of Ex 8.)
Step 1 — detect the trap. Rows ( 1 , 2 , 3 ) and ( 2 , 4 , 6 ) are proportional → rank 1 < 2 → not full row rank → cell C-twist-f → SVD required . Note A = A 8 ⊤ (the transpose of Example 8's matrix), so we can reuse everything.
Why this step? Fat shape is necessary but not sufficient ; we need independent rows.
Step 2 — write A as one outer product. A = ( 1 2 ) ( 1 2 3 ) , i.e. A = c r ⊤ with column-direction c = ( 1 , 2 ) and row-direction r = ( 1 , 2 , 3 ) .
Why this step? Same rank-1 reading as before, now with the roles of the two directions swapped.
Step 3 — unit singular vectors and σ 1 . u = ∥ c ∥ c = 5 1 ( 1 , 2 ) (left, size 2), v = ∥ r ∥ r = 14 1 ( 1 , 2 , 3 ) (right, size 3), and σ 1 = ∥ c ∥∥ r ∥ = 5 ⋅ 14 = 70 .
Why this step? Identical logic to Ex 8; here u lives in R 2 and v in R 3 .
Step 4 — assemble A + . A + = σ 1 1 v u ⊤ = 70 1 ⋅ 14 1 1 2 3 ⋅ 5 1 ( 1 2 ) = 70 1 1 2 3 2 4 6 .
Why this step? A + is 3 × 2 as required, and it equals ( A 8 + ) ⊤ — a good sanity check, since ( A ⊤ ) + = ( A + ) ⊤ always.
Verify (Penrose condition 1): A A + A = A . Here A A + = u u ⊤ = 5 1 ( 1 2 2 4 ) (column-space projection onto the line ( 1 , 2 ) ). Then ( A A + ) A : for row 1, 5 1 [ 1 ⋅ ( 1 , 2 , 3 ) + 2 ⋅ ( 2 , 4 , 6 ) ] = 5 1 ( 5 , 10 , 15 ) = ( 1 , 2 , 3 ) ✓; row 2 gives ( 2 , 4 , 6 ) ✓.
Common mistake The one trap this page drills hardest
"Tall ⇒ use ( A ⊤ A ) − 1 A ⊤ ; fat ⇒ use A ⊤ ( A A ⊤ ) − 1 ." Examples 8 and 9 kill both halves: shape is necessary but you also need independent columns (tall) or independent rows (fat) — i.e. full rank . If any column/row is a combination of others, A ⊤ A (or A A ⊤ ) is singular and you must fall back to the Singular Value Decomposition . Always check rank before choosing a formula.
Best single number fitting x = 3 and x = 1 (Ex 1)? x = 2 , the average; residual ( 1 , − 1 ) ⟂ column ( 1 , 1 ) .
Min-norm solution of x 1 + 2 x 2 = 4 (Ex 3)? x = ( 4/5 , 8/5 ) , perpendicular to nullspace direction ( 2 , − 1 ) .
Why does a tall matrix sometimes still need SVD (Ex 8)? If its columns are dependent (rank deficient), A ⊤ A is singular so the tall formula fails.
Why does a fat matrix sometimes still need SVD (Ex 9)? If its rows are dependent (rank deficient), A A ⊤ is singular so the fat formula fails.
Pseudoinverse of ( 1 1 1 1 ) (Ex 5)? 4 1 ( 1 1 1 1 ) ; note A + A is a projection, not I .
Pseudoinverse of the zero matrix (Ex 6)? The zero matrix itself.
Best-fit resistance from V = 2.1 , 3.9 , 6.1 at I = 1 , 2 , 3 (Ex 7)? R = 28.2/14 ≈ 2.01 Ω .
Handy identity linking transpose and pseudoinverse (Ex 9)? ( A ⊤ ) + = ( A + ) ⊤ .
Check shape and rank first