Noesis
4.5.39Linear Algebra (Full)

Quadratic forms — positive definite, negative definite, indefinite

1,882 words9 min readdifficulty · medium3 backlinks
🇮🇳 Yeh note Hinglish mein padho →

1. What is a quadratic form?

WHY symmetric? Any matrix BB gives x⊤Bx\mathbf{x}^\top B\mathbf{x}, but only the symmetric part matters: x⊤Bx=x⊤B+B⊤2⏟symmetricx+x⊤B−B⊤2⏟antisymmetricx.\mathbf{x}^\top B \mathbf{x} = \mathbf{x}^\top \underbrace{\tfrac{B+B^\top}{2}}_{\text{symmetric}}\mathbf{x} + \mathbf{x}^\top \underbrace{\tfrac{B-B^\top}{2}}_{\text{antisymmetric}}\mathbf{x}. The antisymmetric part contributes 00 (a scalar equals its own transpose, and the antisymmetric piece transposes to its negative). So we always choose AA symmetric — it's unique.

2. The four definiteness types

Figure — Quadratic forms — positive definite, negative definite, indefinite

3. Eigenvalue test — the master derivation (HOW & WHY)

Derivation from scratch. Substitute A=QΛQ⊤A=Q\Lambda Q^\top: Q(x)=x⊤QΛQ⊤x.Q(\mathbf{x}) = \mathbf{x}^\top Q\Lambda Q^\top \mathbf{x}. Let y=Q⊤x\mathbf{y} = Q^\top \mathbf{x} (rotated coordinates). Since QQ is invertible, x≠0  ⟺  y≠0\mathbf{x}\neq0 \iff \mathbf{y}\neq0. Then Q(x)=y⊤Λy=∑i=1nλi yi2.Q(\mathbf{x}) = \mathbf{y}^\top \Lambda \mathbf{y} = \sum_{i=1}^n \lambda_i\, y_i^2.

This is diagonal — pure squares, no cross terms! Now read off the sign:

WHY it works: yi2≥0y_i^2\ge0 always. If every λi>0\lambda_i>0, every term is positive (unless all yi=0y_i=0, i.e. x=0\mathbf{x}=0). If one λ\lambda is ++ and one is −-, pick y\mathbf{y} along each eigenvector to get either sign → indefinite.

4. Leading principal minors test (Sylvester's criterion)

Eigenvalues can be painful by hand. Sylvester's criterion lets you check PD with determinants only.

5. Worked examples

6. Forecast-then-Verify drill

7. The geometric payoff (WHY we care)

Flashcards

What matrix does a quadratic form use, and what property must it have?
Q(x)=x⊤AxQ(\mathbf{x})=\mathbf{x}^\top A\mathbf{x} with AA symmetric (A⊤=AA^\top=A).
How do you build AA from a formula like ax2+bxy+cy2ax^2+bxy+cy^2?
Diagonals a,ca,c; off-diagonals each equal b/2b/2.
Why does the antisymmetric part of a matrix not affect x⊤Bx\mathbf{x}^\top B\mathbf{x}?
A scalar equals its transpose; the antisymmetric part transposes to its negative, forcing it to be 00.
Eigenvalue test for positive definite?
All eigenvalues λi>0\lambda_i > 0.
Eigenvalue test for indefinite?
Eigenvalues have mixed signs (at least one >0>0 and one <0<0).
After diagonalizing A=QΛQ⊤A=Q\Lambda Q^\top, what does Q(x)Q(\mathbf{x}) become?
∑iλiyi2\sum_i \lambda_i y_i^2 where y=Q⊤x\mathbf{y}=Q^\top\mathbf{x}.
Sylvester's criterion for PD?
All leading principal minors Dk>0D_k>0.
Sylvester's criterion for ND?
Signs alternate: (−1)kDk>0(-1)^kD_k>0, i.e. D1<0,D2>0,D3<0,…D_1<0, D_2>0, D_3<0,\dots.
What distinguishes PSD from PD?
PSD allows Q(x)=0Q(\mathbf{x})=0 for some x≠0\mathbf{x}\neq0 (a zero eigenvalue); PD is strictly >0>0.
Hessian PD at a critical point means what?
Local minimum (bowl shape).
Hessian indefinite means?
Saddle point.
Recall Feynman: explain to a 12-year-old

Imagine a machine where you put in arrows and it gives you a number. A positive-definite machine is like a valley: no matter which direction you step away from the bottom, you go up — the number is always positive. A negative-definite one is a hilltop: every step goes down. An indefinite one is a horse-saddle: step one way and you go up, step the other way and you go down. To know which kind you have, we secretly turn our head (rotate the axes) until the machine becomes super simple — just λ1(first)2+λ2(second)2+…\lambda_1 (\text{first})^2 + \lambda_2(\text{second})^2 + \dots. Then we just look at the signs of those λ\lambda numbers: all plus = valley, all minus = hill, mixed = saddle. Easy!

Connections

  • Spectral Theorem — guarantees A=QΛQ⊤A=Q\Lambda Q^\top, the engine of the eigenvalue test.
  • Eigenvalues and Eigenvectors — the signs that classify everything.
  • Symmetric Matrices — why definiteness is even well-defined.
  • Hessian Matrix / Second Derivative Test — the optimization payoff.
  • Determinants — leading principal minors in Sylvester's criterion.
  • Cholesky Decomposition — exists   ⟺  \iff matrix is PD (a practical PD test).
  • Least Squares — normal-equation matrix A⊤AA^\top A is always PSD.

Concept Map

requires

diagonal = square coeff

off-diag = half cross coeff

Spectral Theorem

rotate coords y = QT x

sign of eigenvalues decides

all lambda gt 0

all lambda lt 0

mixed signs

lambda ge 0 with zero

underlies

Quadratic form Q x = xT A x

Symmetric matrix A

Build A from formula

A = Q Lambda QT

Sum of lambda yi squared

Definiteness type

Positive definite bowl min

Negative definite dome max

Indefinite saddle

Semidefinite

2nd-derivative test and stability

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho, quadratic form ka matlab hai ek aisa function Q(x)=x⊤AxQ(\mathbf{x})=\mathbf{x}^\top A\mathbf{x} jisme sirf degree-2 terms hote hain — jaise x2x^2, xyxy, y2y^2. Yahan AA hamesha symmetric matrix lete hain (diagonal pe square ke coefficients, off-diagonal pe cross term ka aadha). Symmetric isliye, kyunki antisymmetric part to apne aap zero ho jaata hai.

Ab asli sawaal: ye machine hamesha positive number deti hai, hamesha negative, ya mixed? Agar hamesha positive (except origin) → positive definite (katore jaisa, minimum). Hamesha negative → negative definite (pahaadi ki choti, maximum). Dono signs aaye → indefinite (ghode ki seat, saddle). Decide kaise karein? Eigenvalues dekho! Spectral theorem se A=QΛQ⊤A=Q\Lambda Q^\top, aur axes ko ghuma ke form ban jaata hai ∑λiyi2\sum \lambda_i y_i^2. Sab λ>0\lambda>0 → PD, sab <0<0 → ND, mixed signs → indefinite. Bas signs check karo.

Haath se jaldi karna ho to Sylvester's criterion use karo — leading principal minors DkD_k. Sab Dk>0D_k>0 → PD. ND ke liye signs alternate karne chahiye: D1<0,D2>0,D3<0…D_1<0, D_2>0, D_3<0\dots — yahin students galti karte hain, sochte hain "sab negative = ND", jo galat hai!

Ye cheez kyun important hai? Optimization mein! Critical point pe Hessian matrix ek quadratic form banati hai. PD → minimum, ND → maximum, indefinite → saddle point. Yani second-derivative test ka pura asli jadu yahi linear algebra hai. Machine learning, stability, least-squares — sab jagah PD matrices ka raj hai.

Go deeper — visual, from zero

D1FoundationsEvery symbol from zero · 1,872 wordsD2Visual walkthroughThe derivation in pictures · 1,823 wordsD3Worked examplesEvery scenario covered · 2,574 wordsD4ExercisesGraded, full solutions · 2,602 wordsD5Question bankConcept traps, quick reveals · 1,516 words

Test yourself — Linear Algebra (Full)

L1 RecognitionL2 RecallL3 ProductionL4 ApplicationL5 Mastery

Connections

Eigenvalues and eigenvectorsAI-ML · 1.1.13Symmetric matrices — spectral theorem (real eigenvalues, orthogonal eigenvectors)Maths · 4.5.38The Hessian matrixAI-ML · 1.2.6Second derivative test — concavity, inflection pointsMaths · 4.1.29Determinants — cofactor expansion along any row - columnMaths · 4.5.21Least squares — normal equations, QR approachMaths · 4.5.41General second-degree equation Ax²+Bxy+Cy²+Dx+Ey+F=0 — discriminant classificationMaths · 3.4.11Critical points — finding, classifyingMaths · 4.4.12Second derivative test — Hessian determinantMaths · 4.4.13