Quadratic forms — positive definite, negative definite, indefinite: Exercises

Before we start, one reference figure fixes all five shapes in your mind — every classification below is really asking "which of these surfaces is it?"

Figure — Quadratic forms — positive definite, negative definite, indefinite

Intuition Reading the reference figure (each colour is a definiteness type)

The figure shows five surfaces $z = Q (x, y)$ , each titled with both its colour and its type, with a black dot at the origin (the critical point). Match colour → type like this:

Blue bowl — positive definite (PD). From the bottom, every direction goes up, so $Q > 0$ around it. (Think "valley".)
Orange dome — negative definite (ND). From the top, every direction goes down, so $Q < 0$ . (Think "hilltop".)
Red saddle — indefinite. One direction goes up, the perpendicular one goes down, so $Q$ takes both signs.
Green trough (flat-bottomed) — positive semidefinite (PSD). A bowl in one direction but perfectly flat along the other — the flat line is the zero-eigenvalue direction, where $Q = 0$ for $x \neq = 0$ .
Gray inverted trough (flat-topped) — negative semidefinite (NSD). A dome in one direction, flat along the other; the mirror image of the green trough. Whenever a solution below says "blue bowl / orange dome / red saddle / green trough / gray trough", it is pointing at this labelled picture.

Level 1 — Recognition

Here $A$ is already diagonal, so $Q = λ_{1} x^{2} + λ_{2} y^{2} + \dots$ and the diagonal entries are the eigenvalues. Just read the signs.

Recall Solution L1.1

WHAT: matrix is $A = (5003)$ , diagonal, so eigenvalues are $5$ and $3$ . WHY: for a diagonal matrix there are no cross terms, so the diagonal numbers are exactly the $λ_{i}$ in $\sum λ_{i} y_{i}^{2}$ . Both $> 0$ ⇒ every input gives a positive output (unless $x = 0$ ). Positive definite (the blue bowl).

Recall Solution L1.2

$A = (- 2 0 0 - 7)$ , eigenvalues $- 2, - 7$ , both $< 0$ . Every output is negative for $x \neq = 0$ . Negative definite (the orange dome).

Recall Solution L1.3

$λ = 1, - 4$ : opposite signs. Feed $(1, 0)$ → $+ 1$ ; feed $(0, 1)$ → $- 4$ . Both signs occur. Indefinite (the red saddle in the figure).

Recall Solution L1.4

$A = (0006)$ , eigenvalues $0, 6$ . Never negative, but $Q (1, 0) = 0$ with $x \neq = 0$ . Positive semidefinite (not PD) — this is the green flat-bottomed trough in the figure. The zero eigenvalue is the whole point: it's the flat direction $x$ -axis.

Recall Solution L1.5

$A = (- 5 0 00)$ , eigenvalues $- 5, 0$ . Never positive (both eigenvalues $\leq 0$ ), but $Q (0, 1) = 0$ with $x \neq = 0$ . Negative semidefinite (NSD) — the gray flat-topped trough. This is the mirror of L1.4: all eigenvalues $\leq 0$ with at least one zero. The surface is a downward trough that is flat along the whole $y$ -axis — a dome in the $x$ -direction, level in the $y$ -direction.

Level 2 — Application

Now the matrices have cross terms. Building $A$ correctly is the first job.

Then use Sylvester's leading principal minors ( $D_{1} = A_{11}$ , $D_{2} = det A$ ) and cross-check with eigenvalues.

Recall Solution L2.1

Build $A$ : diagonals $3, 3$ ; the $x y$ -coefficient is $2$ , and (as shown above) the off-diagonal is half of it because the cross term is counted twice, so $A_{12} = A_{21} = \frac{2}{2} = 1$ . Thus $A = (3113)$ . Sylvester (WHY it certifies PD): by the pivot fact above, $D_{1} > 0$ and $D_{2} > 0$ together mean both pivots $d_{1} = D_{1} = 3$ and $d_{2} = D_{2} / D_{1} = 8/3$ are positive, so $Q$ is a sum of positive squares — the very definition of PD. Compute: $D_{1} = 3 > 0$ ; $D_{2} = 9 - 1 = 8 > 0$ . All positive ⇒ PD. Cross-check eigenvalues: $(3 - λ)^{2} - 1 = 0 \Rightarrow λ = 2, 4$ , both $> 0$ . ✓

Recall Solution L2.2

Build $A$ : diagonals $- 4, - 4$ ; the $x y$ -coefficient is $4$ , halved (counted twice) gives off-diagonal $\frac{4}{2} = 2$ . So $A = (- 4 2 2 - 4)$ . Sylvester for ND — WHY the pattern $D_{1} < 0, D_{2} > 0$ : $A$ is ND exactly when $- A$ is PD. PD requires all leading minors of $- A$ positive. But scaling a $k \times k$ block by $- 1$ multiplies its determinant by $(- 1)^{k}$ , so the $k$ -th minor of $- A$ equals $(- 1)^{k} D_{k}$ . Demanding each $(- 1)^{k} D_{k} > 0$ gives $D_{1} < 0$ (from $k = 1$ : $- D_{1} > 0$ ) and $D_{2} > 0$ (from $k = 2$ : $D_{2} > 0$ ) — the alternating pattern. Check: $D_{1} = - 4 < 0$ ✓; $D_{2} = 16 - 4 = 12 > 0$ ✓. ND. Eigenvalues: $(- 4 - λ)^{2} - 4 = 0 \Rightarrow - 4 - λ = \pm 2 \Rightarrow λ = - 2, - 6$ , both $< 0$ . ✓

Recall Solution L2.3

Build $A$ : diagonals $1, 1$ ; $x y$ -coefficient $6$ , halved ⇒ off-diagonal $3$ . So $A = (1331)$ . $D_{1} = 1 > 0$ but $D_{2} = 1 - 9 = - 8 < 0$ . Not PD (need $D_{2} > 0$ ), and $D_{1} < 0$ fails the ND pattern too. A negative $D_{2} = det A = λ_{1} λ_{2}$ means the eigenvalues have opposite signs. Indefinite. Check: $λ = 4, - 2$ . ✓

Recall Solution L2.4

Build $A$ : diagonals $2, 2, 2$ ; only cross term is $2 x y$ , halved ⇒ $A_{12} = A_{21} = 1$ ; no $x z$ or $y z$ , so those off-diagonals are $0$ . Thus $A = 210120002$ . $D_{1} = 2 > 0$ ; $D_{2} = det (2112) = 3 > 0$ ; $D_{3} = det A$ . WHY $D_{3} = 2 \cdot D_{2}$ (cofactor expansion along the last column): a determinant can be expanded along any row or column — you march down that line, multiply each entry by the smaller determinant left after deleting its row and column (its "minor"), and attach the checkerboard sign $(- 1)^{i + j}$ . The third column here is $(0, 0, 2)^{⊤}$ , so the only surviving term is the $2$ in position $(3, 3)$ : sign $(- 1)^{3 + 3} = + 1$ , and deleting row 3 & column 3 leaves exactly the top-left $2 \times 2$ block whose determinant is $D_{2}$ . Hence $D_{3} = 2 \cdot D_{2} = 2 \cdot 3 = 6 > 0$ . All three $> 0$ ⇒ PD.

Level 3 — Analysis

Now $A$ contains a parameter. Find the range of the parameter for a required type. This is where Sylvester earns its keep.

Recall Solution L3.1

PD via Sylvester: need $D_{1} = 2 > 0$ (always true) and $D_{2} = 16 - k^{2} > 0$ . $16 - k^{2} > 0 ⟺ k^{2} < 16 ⟺ - 4 < k < 4$ . Answer: $- 4 < k < 4$ . At $k = \pm 4$ , $D_{2} = 0$ ⇒ PSD boundary; beyond, indefinite.

Recall Solution L3.2

Build $A$ : the $x y$ -coefficient is $2 k$ , halved ⇒ off-diagonal $k$ . So $A = (1 k k 4)$ , $D_{2} = 4 - k^{2} = λ_{1} λ_{2}$ . Indefinite ⇔ eigenvalues opposite signs ⇔ $D_{2} < 0$ : $4 - k^{2} < 0 ⟺ k^{2} > 4 ⟺ k < - 2 or k > 2$ . Answer: $∣ k ∣ > 2$ .

Recall Solution L3.3

ND needs $D_{1} = k < 0$ and $D_{2} = k^{2} - 1 > 0$ (the alternating pattern from L2.2). $k^{2} - 1 > 0 ⟺ k < - 1$ or $k > 1$ . Intersect with $k < 0$ : $k < - 1$ . Answer: $k < - 1$ . (Eigenvalues $k \pm 1$ , both negative exactly when $k < - 1$ . ✓)

Level 4 — Synthesis

Combine the form with its origin (a formula, a Hessian, a completed square). Multiple tools in one problem.

Recall Solution L4.1

Step 1 — critical point check. $f_{x} = 3 x^{2} - 3 = 0 \Rightarrow x = \pm 1$ ; $f_{y} = 2 y = 0 \Rightarrow y = 0$ . So $(1, 0)$ is critical. ✓ Step 2 — Hessian. $f_{xx} = 6 x$ , $f_{y y} = 2$ , $f_{x y} = 0$ . At $(1, 0)$ : $H = (6002)$ . Step 3 — classify $H$ . Eigenvalues $6, 2 > 0$ ⇒ PD ⇒ local minimum (the blue bowl). Bonus — the other critical point $(- 1, 0)$ : $H = (- 6 0 02)$ , $λ = - 6, 2$ mixed ⇒ indefinite ⇒ saddle (the red surface). This is the whole Second Derivative Test.

Recall Solution L4.2

WHAT (complete the square in $x$ ): $x^{2} + 4 x y + 5 y^{2} = (x + 2 y)^{2} - 4 y^{2} + 5 y^{2} = (x + 2 y)^{2} + y^{2} .$ WHY it proves PD: a sum of two squares is $\geq 0$ , and it equals $0$ only when $x + 2 y = 0$ and $y = 0$ , i.e. $x = y = 0$ . So $Q > 0$ for all $x \neq = 0$ ⇒ PD. Implied eigenvalue signs: PD ⇒ both eigenvalues $> 0$ (indeed $D_{1} = 1 > 0$ , $D_{2} = 5 - 4 = 1 > 0$ , so $λ_{1} λ_{2} = 1 > 0$ and $λ_{1} + λ_{2} = 6 > 0$ : both positive). This completing-the-square factorisation is exactly the idea behind Cholesky Decomposition — PD is precisely the condition that lets you write $A = L L^{⊤}$ .

Recall Solution L4.3

$D_{1} = 9 > 0$ always. Switch happens when $D_{2} = 9 k - 36$ changes sign: $9 k - 36 = 0 \Rightarrow k = 4$ .

$k > 4$ : $D_{2} > 0$ ⇒ PD.
$k < 4$ : $D_{2} < 0$ ⇒ indefinite.
At $k = 4$ : $D_{2} = det A = 0$ ⇒ one eigenvalue is $0$ , the other ( $trace = 13 > 0$ ) positive ⇒ PSD. The knife-edge value is $k = 4$ , type there = PSD.

Level 5 — Mastery

Prove structural facts. These are the "why the tests are even true" problems.

Recall Solution L5.1

Step 1 — rotate to diagonal. By the Spectral Theorem, $A = Q Λ Q^{⊤}$ . Set $y = Q^{⊤} x$ . Because $Q$ is orthogonal it preserves length, so $∥ y ∥ = ∥ x ∥ = 1$ , meaning $\sum y_{i}^{2} = 1$ . Step 2 — rewrite. $Q (x) = \sum_{i} λ_{i} y_{i}^{2}$ . This is a weighted average of the eigenvalues with weights $y_{i}^{2} \geq 0$ summing to $1$ . Step 3 — bound the average. Any weighted average lies between the smallest and largest ingredient: $λ_{m i n} = λ_{m i n} \sum y_{i}^{2} \leq \sum λ_{i} y_{i}^{2} \leq λ_{m a x} \sum y_{i}^{2} = λ_{m a x} .$ Step 4 — definiteness follows. If $λ_{m i n} > 0$ then $Q (x) \geq λ_{m i n} > 0$ for all unit $x$ (hence all $x \neq = 0$ by scaling) ⇒ PD. If $λ_{m a x} < 0$ ⇒ ND. If $λ_{m i n} < 0 < λ_{m a x}$ , the bounds straddle zero and the eigenvector directions realize both signs ⇒ indefinite. $■$

Recall Solution L5.2

WHAT we must show: $x^{⊤} B x > 0$ for all $x \neq = 0$ . Step 1 — substitute $B = C^{⊤} A C$ and regroup: $x^{⊤} B x = x^{⊤} C^{⊤} A C x = (C x)^{⊤} A (C x) .$ Here we used $(C x)^{⊤} = x^{⊤} C^{⊤}$ to fold the two outer factors into one vector. Step 2 — name the inner vector. Let $w = C x$ . Then $x^{⊤} B x = w^{⊤} A w .$ Step 3 — WHY invertibility matters. We started with $x \neq = 0$ . Since $C$ is invertible, $C x = 0$ would force $x = C^{- 1} 0 = 0$ , a contradiction. Hence $w = C x \neq = 0$ . Step 4 — apply PD of $A$ . Because $A$ is PD and $w \neq = 0$ , we have $w^{⊤} A w > 0$ . Therefore $x^{⊤} B x = w^{⊤} A w > 0 for all x \neq = 0 .$ So $B$ is positive definite. $■$ This is why a change of variables (as in Least Squares, where the normal-equation matrix is $C^{⊤} C$ ) keeps a form positive definite.

Recall Solution L5.3

Sylvester: $D_{1} = 2 > 0$ . $D_{2} = det (2 - 1 - 1 2) = 4 - 1 = 3 > 0$ . $D_{3} = det A$ : expand along top row $= 2 (4 - 1) - (- 1) (- 2 - 0) = 6 - 2 = 4 > 0$ . All $> 0$ ⇒ PD. Eigenvalues: for this tridiagonal matrix they are $2 - 2 cos \frac{k π}{4}$ for $k = 1, 2, 3$ , i.e. $2 - 2, 2, 2 + 2$ — all positive. ✓ PD confirmed.