Pseudoinverse: Exercises — Hinglish deep dive

4.5.42 · D4 · Maths › Linear Algebra (Full) › Pseudoinverse

Shuru karne se pehle, teen reminders jinpar tum baar baar rely karoge:

Definition Notation aur terms jo tum poore time use karoge

$I_{k}$ ka matlab hai $k \times k$ identity matrix — diagonal par 1's, baaki jagah 0's; yeh kisi bhi vector ko unchanged chhod deta hai. Jab size clear ho to main sirf $I$ likhta hoon.
Independent columns/rows ka matlab hai inme se koi bhi column/row doosron ke combination se nahi ban sakti — koi redundant nahi hai.
$A$ ki rank $r$ independent columns ki sankhya hai (equivalently, independent rows ki), aur yeh nonzero singular values ki sankhya ke barabar hai. "Full column rank" ka matlab $r = n$ ; "full row rank" ka matlab $r = m$ .
$A$ ka nullspace un sabhi vectors $v$ ka set hai jinhe $A$ zero mein crush kar deta hai, yaani $A v = 0$ . Yeh "invisible" directions hain: kisi bhi solution of $A x = b$ mein koi bhi nullspace vector add karne se $x$ toh badal jaata hai lekin $A x = b$ nahi. Nullspace exactly woh wasted-energy directions hain jinhe minimum-norm solution avoid karta hai.
Kisi scalar function $E (x)$ ka gradient $\nabla E$ har coordinate direction mein uski slopes ka vector hota hai — yeh "uphill" point karta hai, aur yeh exactly ek flat spot par (ek bowl-shaped function ke liye minimum par) zero vector hota hai.
Chaar Penrose conditions jo $A^{+}$ ko uniquely pin down karti hain, order mein:
1. $A A^{+} A = A$
2. $A^{+} A A^{+} = A^{+}$
3. $(A A^{+})^{⊤} = A A^{+}$ (to $A A^{+}$ symmetric hai)
4. $(A^{+} A)^{⊤} = A^{+} A$ (to $A^{+} A$ symmetric hai)
Jab main neeche "Condition 1/2/3/4" kehta hoon, to mera matlab exactly yehi hai.

Level 1 — Recognition

L1.1 — Kaunsa formula?

Har matrix ke liye, uski shape batao, kya uski full column rank hai, full row rank hai, ya koi bhi nahi, aur tum teeno formulas mein se kaunsa use karoge.

$A = 103210$
$B = (100121)$
$C = (1224)$

Recall Solution

$A$ $3 \times 2$ hai (tall). Iske do columns $(1, 0, 3)$ aur $(2, 1, 0)$ ek doosre ke multiples nahi hain, to yeh independent hain → full column rank ( $r = 2 = n$ ). Use karo $A^{+} = (A^{⊤} A)^{- 1} A^{⊤}$ .

$B$ $2 \times 3$ hai (fat). Iske do rows $(1, 0, 2)$ aur $(0, 1, 1)$ independent hain → full row rank ( $r = 2 = m$ ). Use karo $B^{+} = B^{⊤} (B B^{⊤})^{- 1}$ .

$C$ $2 \times 2$ hai (square) lekin row 2 $= 2 \times$ row 1, to rank $r = 1$ hai, full nahi. Na $C^{⊤} C$ aur na $C C^{⊤}$ invertible hai → zaroor SVD formula $C^{+} = V Σ^{+} U^{⊤}$ use karna hoga.

L1.2 — Sach ya jhooth

Agar $A$ square aur invertible hai to $A^{+} = A^{- 1}$ .
$A^{+} A$ hamesha identity ke barabar hota hai.
Matrix $A A^{+}$ symmetric hoti hai.

Recall Solution

Sach — pseudoinverse inverse ko generalise karta hai; invertible square $A$ ke liye dono ek hi hain.
Jhooth — $A^{+} A = I_{n}$ sirf full column rank ke liye. Aam taur par $A^{+} A$ row space par ek orthogonal projection hai, aur ek projection ki eigenvalue $0$ ho sakti hai.
Sach — yeh Penrose Condition 3 hai: $(A A^{+})^{⊤} = A A^{+}$ .

Level 2 — Application

L2.1 — Ek tall pseudoinverse compute karo

$A = 111012$ ke liye $A^{+}$ nikalo, phir ise use karke least-squares problem $A x = b$ solve karo jahan $b = 600$ ho.

Recall Solution

Shape check. $3 \times 2$ , columns independent → Case 1, $A^{+} = (A^{⊤} A)^{- 1} A^{⊤}$ .

Step 1 — $A^{⊤} A$ . ( $A^{⊤}$ mein $A$ ke columns uski rows ban jaate hain.) $A^{⊤} A = (101112) 111012 = (3335) .$ Step 2 — invert karo. Determinant $= 3 \cdot 5 - 3 \cdot 3 = 6$ . $2 \times 2$ matrix $(a c b d)$ ka inverse $\frac{1}{a d - b c} (d - c - b a)$ hota hai: $(A^{⊤} A)^{- 1} = \frac{1}{6} (5 - 3 - 3 3) .$ Step 3 — $A^{⊤}$ se multiply karo. $A^{+} = \frac{1}{6} (5 - 3 - 3 3) (101112) = \frac{1}{6} (5 - 3 20 - 1 3) .$ Step 4 — solve karo. $x = A^{+} b = \frac{1}{6} (5 - 3 20 - 1 3) 600 = \frac{1}{6} (30 - 18) = (5 - 3) .$ To $(0, 6), (1, 0), (2, 0)$ se guzarne wali best-fit line $y = 5 - 3 t$ hai. Koi bhi line teeno points ko hit nahi kar sakti, isliye yeh least-squares answer hai — woh jo $∥ A x - b ∥^{2}$ ko sabse chota banata hai.

L2.2 — Ek fat pseudoinverse compute karo

$B = (111)$ ke liye $B^{+}$ nikalo aur $B x = (3)$ ka minimum-norm solution nikalo.

Recall Solution

Shape check. $1 \times 3$ (fat), ek nonzero row → full row rank → Case 2, $B^{+} = B^{⊤} (B B^{⊤})^{- 1}$ .

Step 1 — $B B^{⊤}$ (ek $1 \times 1$ number): $1 \cdot 1 + 1 \cdot 1 + 1 \cdot 1 = 3$ . Step 2 — invert karo: $(B B^{⊤})^{- 1} = \frac{1}{3}$ . Step 3 — assemble karo: $B^{+} = 111 \cdot \frac{1}{3} = 1/3 1/3 1/3 .$ Step 4 — solve karo: $x = B^{+} (3) = 111 .$ Kyun yeh minimum-norm wala hai. $x_{1} + x_{2} + x_{3} = 3$ ka har solution $(1, 1, 1)$ plus $B$ ke nullspace mein koi vector likh ke likha ja sakta hai — woh directions jinhe $B$ zero mein crush karta hai, yaani sabhi $v$ jahan $v_{1} + v_{2} + v_{3} = 0$ (jaise $(1, - 1, 0)$ ). Aisa $v$ add karne se $B x$ unchanged rehta hai lekin $x$ sirf lamba hota hai, kyunki $(1, 1, 1)$ har nullspace vector ke perpendicular hai: $(1, 1, 1) \cdot (1, - 1, 0) = 0$ . Isliye $(1, 1, 1)$ sabse chhota hai — woh jisme koi nullspace component waste nahi hoti.

Level 3 — Analysis

L3.1 — SVD se rank-deficient matrix

$A = (3000)$ ke liye SVD formula use karke $A^{+}$ compute karo, aur verify karo $A A^{+} A = A$ .

Recall Solution

SVD padho. $A$ pehle se diagonal hai, isliye $U = V = I_{2}$ ( $2 \times 2$ identity), $Σ = (3000)$ . Singular values hain $σ_{1} = 3$ , $σ_{2} = 0$ (to rank $r = 1$ ).

$Σ^{+}$ banao. Nonzero ko flip karo, zero ko zero chhodo: $Σ^{+} = (1/3 0 00)$ . $A^{+} = V Σ^{+} U^{⊤} = (1/3 0 00) .$ Condition 1 verify karo ( $A A^{+} A = A$ ). $A A^{+} = (1000)$ (pehle axis par projection), to $A A^{+} A = (1000) (3000) = (3000) = A$ ✓. Zero wali direction dead rehti hai: wahan invert karne ke liye koi information nahi hai.

L3.2 — Projection viewpoint

L2.1 wale tall $A$ ke liye, $P = A A^{+}$ compute karo aur confirm karo ki yeh (a) symmetric hai aur (b) idempotent hai ( $P^{2} = P$ ). $P$ geometrically kya karta hai?

Recall Solution

$A^{+} = \frac{1}{6} (5 - 3 20 - 1 3)$ . Tab $P = A A^{+} = 111012 \frac{1}{6} (5 - 3 20 - 1 3) = \frac{1}{6} 52 - 1 222 - 1 25 .$ (a) Symmetric: matrix apne transpose ke barabar hai (off-diagonals check karo: $2 = 2$ , $- 1 = - 1$ ) ✓ — yeh Penrose Condition 3 hai. (b) Idempotent — prove kiya, sirf check nahi. "Idempotent" ka matlab hai $P$ do baar apply karna ek baar jaisa hi hai: $P^{2} = P$ . Yeh kyun zaroor hona chahiye kisi bhi $A A^{+}$ ke liye jo genuine pseudoinverse se bana ho, sirf Penrose conditions use karke: $P^{2} = (A A^{+}) (A A^{+}) = A Condition 2 (A^{+} A A^{+}) = A A^{+} = P .$ Middle grouping ne Condition 2 use ki ( $A^{+} A A^{+} = A^{+}$ ). To is form ka har orthogonal projection $P^{2} = P$ satisfy karta hai — already projected vector ko dobara project karne se kuch nahi badaltha. (Is specific $P$ ke liye ek direct numeric check bhi VERIFY mein hai.) Geometry. $P$ $A$ ke column space par ek orthogonal projection hai — 3D ke andar ek 2D plane. Neeche figure mein exactly yahi dikhaya gaya hai: blue plane column space hai (sabhi vectors $A x$ ), peela arrow $b = (6, 0, 0)$ hai, aur $P$ $b$ ko seedha neeche plane par green arrow $P b = A x = (5, 2, - 1)$ par gira deta hai. Lal segment bacha hua residual $b - P b$ hai, aur yeh plane se right angle par milta hai. Woh right angle hi least-squares condition hai $A^{⊤} (b - A x) = 0$ : error tab sabse chota hota hai jab woh plane ke perpendicular ho.

Figure L3.2 — Least-squares as projection. Blue plane: $A$ ka column space (har possible $A x$ ). Yellow arrow $b$ : woh target jahan hum pahunch nahi sakte. Green arrow $P b = A x = (5, 2, - 1)$ : plane mein sabse kareeb point, $A A^{+}$ se produce hota hai. Red segment: residual $b - P b$ , plane ke perpendicular — yahi perpendicularity "least squares" ka matlab hai.

Level 4 — Synthesis

L4.1 — $A^{+}$ do tarike se banao aur compare karo

$A = 200030$ ke liye, $A^{+}$ (i) tall formula se aur (ii) SVD formula se compute karo. Dikhao ki dono agree karte hain.

Recall Solution

(i) Tall formula. $A^{⊤} A = (4009)$ , inverse $(1/4 0 0 1/9)$ . $A^{+} = (A^{⊤} A)^{- 1} A^{⊤} = (1/4 0 0 1/9) (200300) = (1/2 0 0 1/3 00) .$ (ii) SVD. $A$ ke nonzero entries pehle se diagonal par singular values hain: $σ_{1} = 2, σ_{2} = 3$ , $U = I_{3}, V = I_{2}$ ke saath. To $Σ = A$ , aur $Σ^{+} = (1/2 0 0 1/3 00) = A^{+} .$ Dono tarike same matrix dete hain ✓. Jab full column rank aur easy SVD dono apply hon, to inhe agree karna hi chahiye — pseudoinverse unique hota hai.

L4.2 — Ek full non-diagonal SVD pseudoinverse

Rank-1 matrix $A = (1111)$ ke liye $A^{+}$ compute karo.

Recall Solution

Step 1 — $A^{⊤} A$ banao aur eigenvalues nikalo. $A^{⊤} A = (2222)$ . Iske eigenvalues $det (A^{⊤} A - λ I_{2}) = (2 - λ)^{2} - 4 = 0$ se solve hote hain, jo $λ = 4$ ya $λ = 0$ deta hai. Singular values square roots hain: $σ_{1} = 4 = 2$ , $σ_{2} = 0 = 0$ (to rank $r = 1$ ).

Step 2 — right singular vectors $v_{i}$ ( $A^{⊤} A$ ke eigenvectors, unit length). $λ = 4$ ke liye: $(2 - 4 2 2 2 - 4) v = 0 \Rightarrow - 2 v_{1} + 2 v_{2} = 0 \Rightarrow$ direction $(1, 1)$ solve karo. Normalise karo (apni length $2$ se divide karo): $v_{1} = \frac{1}{2} (1, 1)$ . $λ = 0$ ke liye: direction $(1, - 1)$ , normalise karo: $v_{2} = \frac{1}{2} (1, - 1)$ . Orthonormal check karo: $v_{1} \cdot v_{1} = \frac{1}{2} (1 + 1) = 1$ , $v_{2} \cdot v_{2} = 1$ , aur $v_{1} \cdot v_{2} = \frac{1}{2} (1 - 1) = 0$ ✓ — to $V = [v_{1} v_{2}]$ orthogonal hai.

Step 3 — left singular vector $u_{1}$ (sirf nonzero $σ$ ke liye hota hai). Rule: $u_{i} = \frac{1}{σ _{i}} A v_{i}$ . $u_{1} = \frac{1}{2} A v_{1} = \frac{1}{2} \cdot \frac{1}{2} (1111) (11) = \frac{1}{2} \cdot \frac{1}{2} (22) = \frac{1}{2} (1, 1) .$ Unit length check karo: $u_{1} \cdot u_{1} = \frac{1}{2} (1 + 1) = 1$ ✓.

Step 4 — $A^{+}$ assemble karo. SVD sum sirf nonzero singular values par chalta hai, to $r = 1$ ke saath: $A^{+} = \frac{1}{σ _{1}} v_{1} u_{1}^{⊤} = \frac{1}{2} \cdot \frac{1}{2} (11) \cdot \frac{1}{2} (11) = \frac{1}{2} \cdot \frac{1}{2} (1111) = (1/4 1/4 1/4 1/4) .$ Condition 1 se sanity check karo: $A A^{+} = (1/2 1/2 1/2 1/2)$ (line $(1, 1)$ par projection), aur $A A^{+} A = A$ ✓.

Level 5 — Mastery

L5.1 — Tall formula se Penrose conditions prove karo

Maano $A$ ki full column rank hai aur $A^{+} = (A^{⊤} A)^{- 1} A^{⊤}$ . Prove karo ki charon Penrose conditions hold karti hain.

Recall Solution

$G = (A^{⊤} A)^{- 1}$ likho. Pehle note karo $G$ symmetric hai: $A^{⊤} A$ symmetric hai (kyunki $(A^{⊤} A)^{⊤} = A^{⊤} A$ ), aur ek symmetric matrix ka inverse symmetric hota hai kyunki $(M^{- 1})^{⊤} = (M^{⊤})^{- 1} = M^{- 1}$ jab $M^{⊤} = M$ . Isliye $G^{⊤} = G$ .

Pehle Condition 4: $A^{+} A = (A^{⊤} A)^{- 1} A^{⊤} A = G (A^{⊤} A) = I_{n}$ . Identity symmetric hai ✓ — aur yeh dikhata hai ki $A^{+}$ ek genuine left inverse hai.

Condition 1: $A A^{+} A = A (A^{+} A) = A I_{n} = A$ ✓.

Condition 2: $A^{+} A A^{+} = (A^{+} A) A^{+} = I_{n} A^{+} = A^{+}$ ✓.

Condition 3: $A A^{+} = A G A^{⊤}$ . Isko transpose karo: $(A G A^{⊤})^{⊤} = A G^{⊤} A^{⊤} = A G A^{⊤}$ ( $G^{⊤} = G$ use karke), to $A A^{+}$ symmetric hai ✓.

Charon hold karti hain, aur kyunki Penrose conditions ek unique matrix define karti hain, yeh sach mein wahi pseudoinverse hai.

L5.2 — Least squares scratch se, phir $A^{+}$ padho

$∥ A x - b ∥^{2}$ minimise karke $x = (A^{⊤} A)^{- 1} A^{⊤} b$ derive karo, explain karo kyun gradient zero set kiya jaata hai, aur is tarah $A^{+}$ identify karo. Phir L2.1 data ke liye numerically verify karo ki $x = (5, - 3)$ ka error $x = (4, - 2)$ ke paas wale point se chota hai.

Recall Solution

Error setup karo. $∥ A x - b ∥^{2}$ ka matlab "residual vector $A x - b$ ki squared length" hai. $∥ v ∥^{2} = v^{⊤} v$ use karke expand karo: $E (x) = (A x - b)^{⊤} (A x - b) = x^{⊤} A^{⊤} A x - 2 x^{⊤} A^{⊤} b + b^{⊤} b .$ Gradient zero kyun set karte hain? Definitions box se yaad karo: gradient $\nabla E$ har coordinate direction mein $E$ ki slopes ka vector hai. Yahan $E$ $x$ ki ek smooth, bowl-shaped (convex) function hai — bahut saari dimensions mein parabola ki tarah. Iska sabse nichla point exactly wahan hai jahan har direction mein slope flat hai, yaani jahan gradient zero vector ho. Yahi least-squares condition hai. $\nabla E = 2 A^{⊤} A x - 2 A^{⊤} b = 0 \Rightarrow A^{⊤} A x = A^{⊤} b .$ Yeh normal equations hain. Full column rank ke saath $A^{⊤} A$ invertible hai: $x = (A^{⊤} A)^{- 1} A^{⊤} b \Rightarrow A^{+} = (A^{⊤} A)^{- 1} A^{⊤} .$ Geometrically: residual $b - A x$ $A$ ke har column ke orthogonal hai, yaani $A^{⊤} (b - A x) = 0$ — wahi equation. Isliye $A A^{+}$ column space par orthogonal projection hai.

Numeric check (L2.1 data, $A = 111012$ , $b = (6, 0, 0)$ ).

Pseudoinverse answer $x = (5, - 3)$ par: $A x = 1 \cdot 5 + 0 \cdot (- 3) 1 \cdot 5 + 1 \cdot (- 3) 1 \cdot 5 + 2 \cdot (- 3) = (5, 2, - 1)$ , to residual $b - A x = (6 - 5, 0 - 2, 0 - (- 1)) = (1, - 2, 1)$ , aur $∥ b - A x ∥^{2} = 1^{2} + (- 2)^{2} + 1^{2} = 1 + 4 + 1 = 6$ .
Paas wale $x = (4, - 2)$ par: $A x = (4, 2, 0)$ , residual $(6 - 4, 0 - 2, 0 - 0) = (2, - 2, 0)$ , aur $∥ b - A x ∥^{2} = 2^{2} + (- 2)^{2} + 0^{2} = 4 + 4 + 0 = 8$ . Kyunki $6 < 8$ , pseudoinverse solution $(5, - 3)$ genuinely chota squared error deta hai — koi bhi doosra choice ise beat nahi kar sakta, kyunki yeh convex bowl $E (x)$ ka unique minimum hai ✓.

Recall Ek-line self-test

Kisi bhi matrix ke liye, mera pehla move ::: uski rank aur shape check karna hai, phir tall / fat / SVD choose karna. $A A^{+}$ geometrically ::: $A$ ke column space par orthogonal projection hai. $A^{+}$ ke SVD sum mein kitne terms hote hain? ::: exactly $r$ , yaani $A$ ki rank (har nonzero singular value ke liye ek term). $A$ ka nullspace ::: un sabhi $v$ ka set hai jahan $A v = 0$ — woh directions jinhe $A$ zero mein crush karta hai.

Connections

Pseudoinverse — woh parent note jinhe yeh exercises drill karte hain.
Singular Value Decomposition — L3, L4, L5 ke peeche ka engine.
Least Squares Regression — L2.1 aur L5.2 ke peeche ki kahani.
Orthogonal Projection — L3.2 mein $A A^{+}$ ka matlab.
Four Fundamental Subspaces — nullspace directions kyun kill ho jaati hain.
Matrix Inverse — special case $A^{+} = A^{- 1}$ .
Eigenvalues and Eigenvectors — $σ_{i}^{2}$ $A^{⊤} A$ ke eigenvalues hain (L4.2 mein use hue).