Pseudoinverse — Hinglish notes

4.5.42 · Maths › Linear Algebra (Full)

Yeh hai kya?

Ek key geometric fact: $A A^{+}$ ek orthogonal projection hai $A$ ke column space par, aur $A^{+} A$ ek orthogonal projection hai $A$ ke row space par.

$A^{+}$ ko first principles se derive karna

Case 1 — Full column rank ( $rank = n \leq m$ , tall/thin)

Hum $A x = b$ solve karna chahte hain. Agar $b$ column space mein nahi hai, to exact solution exist nahi hoti — toh hum error $∥ A x - b ∥^{2}$ minimize karte hain (least squares).

Case 2 — Full row rank ( $rank = m \leq n$ , short/fat)

Ab $A x = b$ ke infinitely many solutions hain. Hum woh solution choose karte hain jiska norm $∥ x ∥$ sabse chhota ho — Kyun? Yeh unique "no wasted energy" solution hai, jo nullspace ke perpendicular hai.

Case 3 — General rank (SVD se) — universal formula

Worked Examples

Common mistake Common errors ko steel-man karna

Mistake 1: " $A^{+} = (A^{⊤} A)^{- 1} A^{⊤}$ hamesha." Kyun sahi lagta hai: yeh formula har least-squares textbook mein hai. Catch: iske liye full column rank chahiye, warna $A^{⊤} A$ singular hai aur invert nahi ho sakta. Fix: rank-deficient matrices ke liye SVD formula use karo.

Mistake 2: " $A^{+} A = I$ hamesha." Kyun sahi lagta hai: ek inverse cheezein undo karta hai. Sach: $A^{+} A = I$ sirf full column rank ke liye hota hai. Generally $A^{+} A$ ek projection hai, identity nahi (iska eigenvalue 0 bhi ho sakta hai).

Mistake 3: "Zero singular values ko bhi invert karo (ek tiny number use karo)." Kyun tempting lagta hai: awkward zero se bachna ho. Danger: $1/ ϵ$ bahut bada ho jaata hai, noise enormously amplify ho jaati hai. Fix: truly zero (ya negligible) $σ_{i}$ ko $0$ par hi map karo.

Recall Feynman: 12-saal ke bacche ko samjhao

Ek vending machine socho. Ek normal inverse ek aisi machine hai jo, snack dene par, exactly wahi coins wapas deti hai jo tune daale the. Lekin kuch machines tuti hoti hain — woh information squish kar deti hain toh coins perfectly wapas nahi mil sakte, ya bahut saari coin-combinations ek hi snack deti hain. Pseudoinverse ek sabse fair refund machine hai: agar perfect refund possible nahi, to woh answer deti hai jo sabse close ho (least error); agar kaafi refunds kaam karte hain, to woh sabse chhota, simplest wala deti hai (least wasted coins). Yeh kabhi crash nahi karti, chahe machine kaisi bhi ho.

Flashcards

Chaar Penrose conditions kya hain?

A A^{+} A = A

;

A^{+} A A^{+} = A^{+}

;

(A A^{+})^{⊤} = A A^{+}

;

(A^{+} A)^{⊤} = A^{+} A

Full column rank ke liye Pseudoinverse?

A^{+} = (A^{⊤} A)^{- 1} A^{⊤}

(ek left inverse,

A^{+} A = I

Full row rank ke liye Pseudoinverse?

A^{+} = A^{⊤} (A A^{⊤})^{- 1}

(ek right inverse,

A A^{+} = I

A^{+}

ke liye universal SVD formula?

A^{+} = V Σ^{+} U^{⊤}

, har nonzero

σ_{i}

ko invert karo, zeros ko zeros rehne do.

A A^{+}

geometrically kya represent karta hai?

A

ke column space par orthogonal projection.

Least squares se

A^{⊤} A x = A^{⊤} b

kyun milta hai?

\nabla∥ A x - b ∥^{2} = 0

set karne par; residual column space ke perpendicular hota hai.

A^{+} = A^{- 1}

kab hota hai?

Jab

A

square aur invertible ho.

Fat-matrix pseudoinverse kaun sa problem solve karta hai?

A x = b

ke infinitely many solutions mein se minimum-norm solution.

Zero singular values ko invert kyun nahi karte?

1/0

undefined hai aur tiny values noise ko blow up kar deti hain; un directions mein koi information nahi hoti.

Connections

Singular Value Decomposition — woh engine jo $A^{+}$ ko universally define karta hai.
Least Squares Regression — full-column-rank case ka application.
Four Fundamental Subspaces — $A^{+}$ column space ↔ row space map karta hai, nullspaces ko kill karta hai.
Orthogonal Projection — $A A^{+}$ aur $A^{+} A$ projection matrices hain.
Matrix Inverse — special case jab $A$ square invertible ho.
Eigenvalues and Eigenvectors — $σ_{i}^{2}$ , $A^{⊤} A$ ke eigenvalues hain.