Quadratic forms — positive definite, negative definite, indefinite

4.5.39 · Maths › Linear Algebra (Full)

1. Quadratic form kya hota hai?

WHY symmetric? Koi bhi matrix $B$ se $x^{⊤} B x$ milta hai, lekin sirf symmetric part matter karta hai: $x^{⊤} B x = x^{⊤} symmetric \frac{B + B ^{⊤}}{2} x + x^{⊤} antisymmetric \frac{B - B ^{⊤}}{2} x .$ Antisymmetric part ka contribution $0$ hota hai (ek scalar apne transpose ke barabar hota hai, aur antisymmetric piece transpose hone par negative ho jata hai). Isliye hum hamesha $A$ ko symmetric choose karte hain — yeh unique hota hai.

2. Chaar definiteness types

Figure — Quadratic forms — positive definite, negative definite, indefinite

3. Eigenvalue test — master derivation (HOW & WHY)

Scratch se derivation. $A = Q Λ Q^{⊤}$ substitute karo: $Q (x) = x^{⊤} Q Λ Q^{⊤} x .$ Maano $y = Q^{⊤} x$ (rotated coordinates). Kyunki $Q$ invertible hai, $x \neq = 0 ⟺ y \neq = 0$ . Tab $Q (x) = y^{⊤} Λ y = \sum_{i = 1}^{n} λ_{i} y_{i}^{2} .$

Yeh diagonal hai — pure squares, koi cross terms nahi! Ab sign padho:

Formula Eigenvalue criterion

$Q (x) = \sum_{i} λ_{i} y_{i}^{2} \Rightarrow$

Sabhi $λ_{i}$ ...	Type
$> 0$	Positive definite
$< 0$	Negative definite
$\geq 0$ (kuch $= 0$ )	PSD
$\leq 0$ (kuch $= 0$ )	NSD
mixed signs	Indefinite

WHY kaam karta hai: $y_{i}^{2} \geq 0$ hamesha. Agar har $λ_{i} > 0$ , to har term positive hai (jab tak sabhi $y_{i} = 0$ na hon, yaani $x = 0$ ). Agar ek $λ$ $+$ hai aur ek $-$ hai, to $y$ ko har eigenvector ke along choose karo aur dono signs milenge → indefinite.

4. Leading principal minors test (Sylvester's criterion)

Eigenvalues haath se nikalna painful ho sakta hai. Sylvester's criterion sirf determinants se PD check karne deta hai.

5. Worked examples

6. Forecast-then-Verify drill

7. Geometric payoff (WHY care karte hain)

Flashcards

What matrix does a quadratic form use, and what property must it have?

Q (x) = x^{⊤} A x

with

A

symmetric (

A^{⊤} = A

How do you build

A

from a formula like

a x^{2} + b x y + c y^{2}

Diagonals

a, c

; off-diagonals each equal

b /2

Why does the antisymmetric part of a matrix not affect

x^{⊤} B x

A scalar equals its transpose; the antisymmetric part transposes to its negative, forcing it to be

0

Eigenvalue test for positive definite?

All eigenvalues

λ_{i} > 0

Eigenvalue test for indefinite?

Eigenvalues have mixed signs (at least one

> 0

and one

< 0

After diagonalizing

A = Q Λ Q^{⊤}

, what does

Q (x)

become?

\sum_{i} λ_{i} y_{i}^{2}

where

y = Q^{⊤} x

Sylvester's criterion for PD?

All leading principal minors

D_{k} > 0

Sylvester's criterion for ND?

Signs alternate:

(- 1)^{k} D_{k} > 0

, i.e.

D_{1} < 0, D_{2} > 0, D_{3} < 0, \dots

What distinguishes PSD from PD?

PSD allows

Q (x) = 0

for some

x \neq = 0

(a zero eigenvalue); PD is strictly

> 0

Hessian PD at a critical point means what?

Local minimum (bowl shape).

Hessian indefinite means?

Saddle point.

Recall Feynman: 12-saal ke bache ko samjhao

Ek aisi machine socho jisme tum arrows daalo aur woh ek number deti hai. Ek positive-definite machine valley jaisi hoti hai: chahe kisi bhi direction mein qadam uthao bottom se, tum upar jaate ho — number hamesha positive hota hai. Ek negative-definite wali hilltop jaisi hoti hai: har qadam neeche jaata hai. Ek indefinite wali horse-saddle jaisi hoti hai: ek taraf jao to upar, doosri taraf jao to neeche. Yeh jaanne ke liye ki kaunsi hai, hum secretly apna sar ghuma lete hain (axes rotate karte hain) jab tak machine super simple na ban jaye — bas $λ_{1} (pehla)^{2} + λ_{2} (doosra)^{2} + \dots$ . Phir sirf un $λ$ numbers ke signs dekho: sab plus = valley, sab minus = hill, mixed = saddle. Easy!

Connections

Spectral Theorem — guarantee karta hai $A = Q Λ Q^{⊤}$ , eigenvalue test ka engine.
Eigenvalues and Eigenvectors — woh signs jo sab classify karte hain.
Symmetric Matrices — kyun definiteness well-defined bhi hai.
Hessian Matrix / Second Derivative Test — optimization ka payoff.
Determinants — Sylvester's criterion mein leading principal minors.
Cholesky Decomposition — exist karta hai $⟺$ matrix PD hai (ek practical PD test).
Least Squares — normal-equation matrix $A^{⊤} A$ hamesha PSD hoti hai.