Quadratic forms — Hinglish notes

1.1.18 · AI-ML › Linear Algebra Essentials

Quadratic form KYA hota hai?

Symmetric KYU? $A$ ka sirf symmetric part hi $Q$ ko affect karta hai. $A = S + K$ split karo jahan $S = \frac{1}{2} (A + A^{⊤})$ symmetric hai aur $K = \frac{1}{2} (A - A^{⊤})$ skew hai. Toh $x^{⊤} K x = 0$ hamesha hota hai (ek scalar apne transpose ke barabar hota hai: $x^{⊤} K x = (x^{⊤} K x)^{⊤} = x^{⊤} K^{⊤} x = - x^{⊤} K x$ , toh yeh 0 hai). Isliye hum hamesha $A$ ko symmetric maante hain — kuch bhi lost nahi hota.

Ise KAISE expand karte hain (scratch se derivation)

Double sum se shuru karo aur dekho har piece kahan se aati hai. $n = 2$ lo: $x^{⊤} A x = [x_{1} x_{2}] [a b b c] [x_{1} x_{2}] .$

Step 1 — $A x$ ka inner multiply karo: $A x = [a x_{1} + b x_{2} b x_{1} + c x_{2}] .$ Yeh step kyun? Matrix–vector product pehle ek dimension collapse karta hai, ek vector bacha ke.

Step 2 — $x^{⊤}$ se dot karo: $Q = x_{1} (a x_{1} + b x_{2}) + x_{2} (b x_{1} + c x_{2}) = a x_{1}^{2} + 2 b x_{1} x_{2} + c x_{2}^{2} .$ Yeh step kyun? Cross terms $b x_{1} x_{2}$ do baar aate hain ( $A_{12}$ aur $A_{21}$ dono se), jisse factor $2 b$ banta hai. Yahi key reading rule hai.

Definiteness — bowl ki shape

Ise KAISE test karte hain — eigenvalues se (derivation). Kyunki $A$ symmetric hai, Spectral Theorem ek orthonormal eigenbasis deta hai $A = Q Λ Q^{⊤}$ jahan $Q^{⊤} Q = I$ . $y = Q^{⊤} x$ substitute karo (ek rotation, $∥ y ∥ = ∥ x ∥$ ): $x^{⊤} A x = x^{⊤} Q Λ Q^{⊤} x = y^{⊤} Λ y = \sum_{i = 1}^{n} λ_{i} y_{i}^{2} .$

Worked examples

Common mistakes

Recall Ek 12-saal ke bachche ko samjhao (Feynman)

Ek landscape imagine karo. Tum origin par khade ho (flat point) aur ek machine tumhe batati hai ki kisi bhi direction mein ek step door ground kitni "oopar" hai. Quadratic form wahi machine hai: use ek direction arrow $x$ do, woh ek height return karta hai. Matrix $A$ secretly store karta hai ki ground kitni steeply bend karti hai. Agar yeh har direction mein upar bend kare, toh tum bowl ke bottom par ho (positive definite). Agar ek taraf upar aur doosri taraf neeche bend kare, toh tum horse saddle par ho (indefinite). Eigenvectors special "downhill/uphill" directions hain, aur eigenvalues batate hain ki wahan kitna sharply bend hota hai.

Recall flashcards

Quadratic form kya hota hai?

Scalar

Q (x) = x^{⊤} A x = \sum_{ij} A_{ij} x_{i} x_{j}

x

ki degree-2 function.

Hum

A

ko hamesha symmetric kyun le sakte hain?

Skew part zero contribute karta hai:

x^{⊤} K x = 0

, isliye sirf

\frac{1}{2} (A + A^{⊤})

matter karta hai.

a x_{1}^{2} + b x_{1} x_{2} + c x_{2}^{2}

se matrix kaise padhte hain?

A = [a b /2 b /2 c]

— cross term ko half karo.

Positive definite ki definition?

x^{⊤} A x > 0

sabhi

x \neq = 0

ke liye.

Definiteness ke liye eigenvalue test?

Eigen-coords mein

Q = \sum λ_{i} y_{i}^{2}

; sabhi

λ_{i} > 0

⟺ positive definite; mixed signs ⟺ indefinite.

Sylvester's criterion (2×2)?

Positive definite ⟺

A_{11} > 0

aur

det A > 0

Completing the square se kya pata chalta hai?

Weighted perfect squares ka sum; positive coefficients ⟺ positive definite (yahi

L D L^{⊤}

hai).

A

ke eigenvectors ka geometric meaning?

Quadratic surface ke principal axes; eigenvalues unke saath curvatures hain.

ML mein definiteness kyun important hai?

Positive-definite Hessian ⟹ convex bowl ⟹ unique minimum ⟹ well-behaved optimization.

Connections

Symmetric matrices — quadratic forms inse define hote hain.
Eigenvalues and eigenvectors — definiteness decide karte hain aur principal axes dete hain.
Spectral theorem — $\sum λ_{i} y_{i}^{2}$ rewrite justify karta hai.
Positive definite matrices — "bowl" wala case.
Convex optimization / Hessian matrix — loss functions ka second-order behaviour.
PCA — $∥ x ∥ = 1$ constraint mein $x^{⊤} Σ x$ maximize karna ek quadratic form problem hai.
Cholesky decomposition — algorithmic completing-the-square.