Pseudoinverse: Question bank — Hinglish deep dive

4.5.42 · D5 · Maths › Linear Algebra (Full) › Pseudoinverse

Shuru karne se pehle, yahan har woh symbol aur fact hai jis par yeh page tika hua hai. Isse ek baar padh lo; neeche ke traps yeh sab assume karte hain.

Recall Quick symbol refresher (shapes, ranks, subspaces)

$A \in R^{m \times n}$ ka matlab hai $A$ mein $m$ rows hain, $n$ columns hain. Tall = rows zyada hain columns se ( $m > n$ ); fat = columns zyada hain rows se ( $m < n$ ).
$A^{⊤}$ transpose hai (diagonal ke across flip karo). $A^{- 1}$ ordinary inverse hai (sirf square, non-singular $A$ ke liye).
Rank = independent columns ki sankhya = independent rows ki sankhya. Full column rank ka matlab hai saare $n$ columns independent hain; full row rank ka matlab hai saare $m$ rows independent hain.
$∥ x ∥$ ek vector ki length hai; $∥ A x - b ∥^{2}$ woh squared error hai jo hum least squares mein minimise karte hain.
Chaar subspaces (dekho Four Fundamental Subspaces): column space (saare outputs $A x$ ), row space, nullspace (woh inputs jinhein $A$ $0$ par bhejta hai), left nullspace.

Definition SVD ingredients

$U$ , $Σ$ , $V$ (in letters ka matlab kya hai) Singular Value Decomposition kisi bhi $A \in R^{m \times n}$ ko $A = U Σ V^{⊤}$ likhta hai jahan:

$U$ ek $m \times m$ orthogonal matrix hai: iske columns $R^{m}$ ka ek orthonormal basis hain (saare mutually perpendicular unit vectors), isliye $U^{⊤} U = U U^{⊤} = I_{m}$ .
$V$ ek $n \times n$ orthogonal matrix hai: iske columns $R^{n}$ ka ek orthonormal basis hain, isliye $V^{⊤} V = V V^{⊤} = I_{n}$ .
$Σ$ ka shape $m \times n$ hai ( $A$ ke shape jaisa hi), har jagah zero hai siwaaye diagonal entries $σ_{1} \geq σ_{2} \geq \dots \geq 0$ ke, jo singular values hain. Nonzero $σ_{i}$ ki sankhya $A$ ki rank ke barabar hoti hai.

$Σ^{+}$ banane ka rule (transpose-and-flip): $Σ$ lo, uska shape $n \times m$ mein transpose karo, aur diagonal par har nonzero $σ_{i}$ ko $\frac{1}{σ _{i}}$ se replace karo jabki har zero ko $0$ hi rehne do. Phir $A^{+} = V Σ^{+} U^{⊤}$ . $Σ = σ_{1} 00 000 (3 \times 2) ⟶ Σ^{+} = (1/ σ_{1} 0 0000) (2 \times 3)$

Neeche ki do pictures woh geometry hain jo is page ka har trap secretly test karta hai: jawab dene se pehle inhe dekh lo.

True or false — justify

A^{+}

har real matrix ke liye exist karta hai, zero matrix bhi include hai.

True. Singular Value Decomposition kisi bhi shape aur rank ke liye

A = U Σ V^{⊤}

deta hai, isliye

A^{+} = V Σ^{+} U^{⊤}

hamesha exist karta hai; zero matrix ka pseudoinverse phir se zero matrix hota hai (invert karne ke liye kuch nahi hai).

Agar

A

square aur invertible hai, toh

A^{+} = A^{- 1}

True. Full rank aur square shape ke saath,

A^{⊤} A

invertible hota hai aur

(A^{⊤} A)^{- 1} A^{⊤} = A^{- 1} (A^{⊤})^{- 1} A^{⊤} = A^{- 1}

; Matrix Inverse pseudoinverse ka ek special case hai.

A^{+} A = I

har matrix

A

ke liye.

False.

A^{+} A = I

sirf tab hota hai jab

A

mein full column rank ho. Generally

A^{+} A

, orthogonal projection onto the row space hota hai, jiska nullspace directions ke saath eigenvalue

0

ho sakta hai.

A A^{+}

aur

A^{+} A

hamesha symmetric hote hain.

True. Penrose conditions 3 aur 4 exactly

(A A^{+})^{⊤} = A A^{+}

aur

(A^{+} A)^{⊤} = A^{+} A

maangti hain; symmetry hi inhe orthogonal projections banati hai, tedhe (skewed) nahin.

(A^{+})^{+} = A

har matrix ke liye.

True. Pseudoinverse do baar lene par original wapas mil jaata hai — chaar Penrose conditions

A

aur

A^{+}

ke roles mein symmetric hain, isliye

A

unhe

A^{+}

ke liye satisfy karta hai (poora "kyun" Edge-cases section mein hai).

A^{+}

hamesha

A

ke saath same shape ka hota hai.

False. Agar

A

ka shape

m \times n

hai, toh

A^{+}

ka shape

n \times m

hota hai — shape transpose ho jaata hai taaki products

A A^{+}

(

m \times m

) aur

A^{+} A

(

n \times n

) ka sense bane.

Ek fat, full-row-rank matrix ke liye,

A A^{+} = I_{m}

hota hai lekin

A^{+} A \neq = I_{n}

True.

A^{+}

yahan ek right inverse hai:

A A^{+} = I_{m}

. Lekin

A^{+} A

sirf row space par projection hai (dimension

m < n

), isliye yeh poora

n \times n

identity nahi ho sakta.

Formula

A^{+} = (A^{⊤} A)^{- 1} A^{⊤}

kisi bhi tall matrix ke liye kaam karta hai.

False. Iske liye full column rank chahiye. Ek tall matrix ke columns phir bhi dependent ho sakte hain, jisse

A^{⊤} A

singular aur non-invertible ho jaata hai — tab SVD formula use karna padega.

Agar

A

ki har singular value nonzero hai, toh

A^{+} A = I

True lekin dhyan se padho. Saari nonzero singular values ka matlab hai full column rank (rank

= n

), jo

A^{+} A = I_{n}

deta hai. Iska matlab

A A^{+} = I

nahin hota jab tak

A

full row rank bhi na ho (yaani square).

Spot the error

A

ka shape

3 \times 2

hai aur rank 1 hai, toh main

A^{+} = (A^{⊤} A)^{- 1} A^{⊤}

use karunga."

Error: 2-column matrix ke liye rank 1 ka matlab hai columns dependent hain, isliye

A^{⊤} A

singular hai — use invert nahi kar sakte. Iske bajaye Singular Value Decomposition se

A^{+} = V Σ^{+} U^{⊤}

use karo.

"Ek singular value

1 0^{- 9}

hai, practically zero, toh main safe rehne ke liye use

1 0^{9}

mein invert kar dunga."

Error: ek tiny

σ

ko invert karne se us direction mein noise massively amplify ho jaata hai. Negligible singular values ko exactly

0

treat karo taaki woh

0

par map hon (parent note mein Mistake 3 dekho).

"Kyunki

A A^{+} A = A

hai, toh right mein se

A

cancel karne par

A^{+} A = I

milta hai."

Error: "cancel karna" matlab dono sides ko ek factor hatane ke liye inverse se multiply karna — lekin

A

ka koi inverse nahi hota jab woh square-and-invertible nahi hota, toh multiply karne ke liye kuch nahi hai.

A A^{+} A = A

ek projection identity hai (

A A^{+}

column space par project karta hai); iska matlab

A^{+} A = I

nahi hota jab tak

A

full column rank na ho.

A^{+}

inverse hai, toh kisi bhi

b

ke liye

A A^{+} b = b

Error:

A A^{+}

column space par orthogonal projection hai, isliye

A A^{+} b = b

sirf tab hoga jab

b

pehle se column space mein ho. Warna yeh column space ka closest point return karta hai, aur exactly yahi reason hai ki least squares ko

A^{+}

ki zaroorat padti hai.

Σ^{+}

nikalne ke liye main

Σ

ki har diagonal entry invert kar dunga."

Error: sirf nonzero singular values ko invert karo; zeros zero hi rehte hain. Shape bhi transpose karni padti hai (ek

m \times n

block

n \times m

ban jaata hai) taaki product

V Σ^{+} U^{⊤}

ke dimensions match karein.

A x = b

ke liye jab infinitely many solutions hain,

A^{+} b

koi bhi ek valid random solution deta hai."

Error:

A^{+} b

woh specific solution deta hai jiska minimum norm

∥ x ∥

ho — woh solution jo poori tarah row space mein rehta hai, nullspace ke orthogonal. Yeh uniquely determined hota hai, arbitrary nahi.

σ_{i}

A^{⊤} A

ke eigenvalues hain."

Error: squares

σ_{i}^{2}

A^{⊤} A

ke eigenvalues hain (dekho Eigenvalues and Eigenvectors). Singular values khud unke non-negative square roots hote hain.

Why questions

Hum least squares mein

∥ A x - b ∥

ki jagah

∥ A x - b ∥^{2}

kyun minimise karte hain?

Squared length

x

ki ek smooth convex function hai, isliye uska gradient simple hai (

2 A^{⊤} A x - 2 A^{⊤} b

) aur exactly ek minimum par zero hota hai; un-squared norm minimise karne par identical answer milta hai lekin expression messier aur zero par non-differentiable hoti hai.

Gradient zero set karne par

A^{⊤} A x = A^{⊤} b

kyun milta hai?

Minimum par residual

A x - b

column space ke orthogonal hona chahiye, yaani

A^{⊤} (A x - b) = 0

. Yahi Least Squares Regression ka geometric core hai: error achievable outputs ke space se seedha bahar point karta hai.

Fat system ke minimum-norm solution ke liye hum

x = A^{⊤} w

kyun likh sakte hain?

x

ko row space (yaani

A^{⊤}

ki range) mein force karne se nullspace ka component hat jaata hai. Nullspace components

A x

ko badle bina length badhaate hain, isliye unhe discard karna exactly wahi hai jo

∥ x ∥

ko smallest banata hai.

SVD ko "universal" pseudoinverse formula kyun kaha jaata hai?

Yeh kabhi

A^{⊤} A

A A^{⊤}

ke invertible hone ki demand nahi karta — yeh kisi bhi shape aur kisi bhi rank ke liye sirf nonzero singular values ko invert karke kaam karta hai, isliye full-column-rank, full-row-rank, aur rank-deficient cases sab ek hi stroke mein cover ho jaate hain.

A A^{+}

generally identity ki jagah projection kyun hota hai?

A A^{+}

sirf column space ke andar outputs tak pahunch sakta hai; agar woh space

R^{m}

se chhota hai, toh kuch directions apne nearest in-space point par flatten ho jaati hain, aur koi bhi map jo directions flatten kare uska eigenvalue

0

hota hai, toh woh

I

nahi ho sakta.

Do special-case formulas

(A^{⊤} A)^{- 1} A^{⊤}

aur

A^{⊤} (A A^{⊤})^{- 1}

SVD formula se kyun agree karte hain?

A = U Σ V^{⊤}

ko kisi bhi formula mein substitute karke aur simplify karke (using

U^{⊤} U = I

V^{⊤} V = I

)

V Σ^{+} U^{⊤}

par collapse ho jaata hai; woh same object hi hain, sirf us case ke liye likhe gaye hain jahan relevant Gram matrix invertible ho.

Least-squares optimum par residual column space ke orthogonal kyun hona chahiye?

Agar residual ka koi component column space ke andar hota, toh hum

x

ko us component cancel karne ke liye move karke error shrink kar sakte the — isliye true minimum par aise koi component nahi bachta, aur residual perpendicular reh jaata hai.

Edge cases

(A^{+})^{+} = A

kyun hota hai, sketch karo.

(A^{+}, A)

ke pair ke liye chaar Penrose conditions

(A, A^{+})

ki jagah likhte hain: condition 1 ban jaati hai

A^{+} A A^{+} = A^{+}

, condition 2 ban jaati hai

A A^{+} A = A

, aur conditions 3–4 ek doosre mein swap ho jaati hain — saari charon phir bhi hold karti hain kyunki woh

(A, A^{+})

ke liye hold karti theen. Kyunki

A

"the pseudoinverse of

A^{+}

" ki chaar defining conditions satisfy karta hai, aur woh matrix unique hai,

A

(A^{+})^{+}

hai.

Jab

A

n \times n

zero matrix hai, toh

A^{+}

kya hai?

n \times n

zero matrix. Har singular value

0

hai, isliye

Σ^{+}

sab zeros hai aur

A^{+} = V Σ^{+} U^{⊤} = 0

— kisi bhi direction mein invert karne ke liye koi information nahi hai.

Ek rectangular zero matrix (

m \times n

) ke liye

0^{+}

kya hai?

n \times m

zero matrix. Shape flip hota hai (hamesha ki tarah) lekin har entry phir bhi

0

hai, kyunki koi bhi direction invertible information carry nahi karta.

Ek rank-deficient square matrix ke liye, kya

A^{+} = A^{- 1}

hota hai?

Nahi.

A^{- 1}

ek singular matrix ke liye exist nahi karta.

A^{+}

SVD ke zariye phir bhi exist karta hai aur sirf row/column spaces par inverse ki tarah act karta hai, nullspace directions ko

0

par map karta hai.

Agar

A

mein ek nullspace hai, toh

A^{+}

A

ke left nullspace ke vectors ka kya karta hai?

Unhe

0

par bhejta hai.

A

ke left-nullspace directions column space ke bahar hain, isliye

A^{+}

(jo sirf column-space directions ko row space mein wapas invert karta hai) ke paas wahan invert karne ke liye kuch nahi hota.

Ek tall matrix ke kuch singular values zero ke barabar hain. Kaun sa formula use karoge, aur kitne zeros ho sakte hain?

SVD formula

A^{+} = V Σ^{+} U^{⊤}

use karo. Zero singular values ki sankhya

n - rank (A)

ke barabar hoti hai, jo exactly nullspace ki dimension hai; har zero ek nullspace direction mark karta hai jo

0

par map ho jaati hai. Ek bhi aisa zero

A^{⊤} A

ko singular bana deta hai, toh tall-matrix formula

(A^{⊤} A)^{- 1} A^{⊤}

fail ho jaata hai.

Agar

A

ek nonzero column vector

v

(

n \times 1

) hai, toh

A^{+}

kya hai?

Row vector

A^{+} = \frac{v ^{⊤}}{∥ v ∥ ^{2}}

. Iska full column rank hai (rank 1), isliye

A^{+} = (v^{⊤} v)^{- 1} v^{⊤}

, aur yeh

v v^{+}

ke zariye

v

ke through line par projection ki tarah act karta hai.

Connections

Singular Value Decomposition — upar ke har edge case ke peeche ka universal engine.
Least Squares Regression — jahan orthogonality-of-residual ka "kyun" rehta hai.
Four Fundamental Subspaces — nullspace aur column-space traps yahin se aate hain.
Orthogonal Projection — explain karta hai kyun $A A^{+}$ aur $A^{+} A$ projections hain, identities nahi.
Matrix Inverse — square-invertible special case.
Eigenvalues and Eigenvectors — $σ_{i}^{2}$ -as-eigenvalues trap.