Quadratic forms: Question bank — AI-ML deep dive

Before we start, one reminder of the objects we keep referring to, so no symbol is unearned:

Two more pieces of vocabulary are used constantly below, so let us earn them before any classification uses them.

Before the traps, a visual atlas of every shape a $2$ -variable quadratic form can take — refer back to it whenever an item mentions "bowl", "saddle", "dome", "trough", "ridge" or "flat". Each panel plots $Q (x_{1}, x_{2})$ as a height surface, its contour lines (level curves), and the eigen-axes (principal directions). Read the panels left-to-right, top-to-bottom: top-left = bowl (positive definite), top-right = dome (negative definite), bottom-left = saddle (indefinite), bottom-right = trough (positive semidefinite, flat along one eigen-axis). The white dashed lines in every panel are the eigenvectors — the principal axes.

Now the contour view. Each panel shows the same surfaces from directly above, so height becomes colour-free level curves. Read them left-to-right, top-to-bottom: top-left = positive definite (closed ellipses hugging the minimum), top-right = indefinite (hyperbolas that open both ways — the saddle's signature), bottom-left = positive semidefinite (parallel straight lines: no bending along the flat eigen-axis), bottom-right = the zero matrix (a single flat plane, $Q = 0$ everywhere). The dashed white lines are again the eigen-axes, labelled with their eigenvalue.

True or false — justify

A $2 \times 2$ symmetric matrix reads off a polynomial as $A = [a b /2 b /2 c]$ for $a x_{1}^{2} + b x_{1} x_{2} + c x_{2}^{2}$ . Keep that picture in mind for every "true/false" below.

The matrix

[1221]

is positive definite because both diagonal entries are positive.

False. Its determinant is

1 - 4 = - 3 < 0

, so the eigenvalues have opposite signs — it is indefinite. Positive diagonal never guarantees positive definiteness because a large off-diagonal can tilt one axis downward.

Q (x) > 0

at the specific point

x = (1, 1)

, then

A

is positive definite.

False. Definiteness demands

Q > 0

for every non-zero

x

; one lucky sample proves nothing, since another direction could give a negative value.

For any square matrix

A

(not necessarily symmetric),

Q (x) = x^{⊤} A x

still equals

x^{⊤} S x

where

S = \frac{1}{2} (A + A^{⊤})

True. The skew part

K = \frac{1}{2} (A - A^{⊤})

satisfies

K^{⊤} = - K

, and

x^{⊤} K x

is a single number so it equals its own transpose:

x^{⊤} K x = (x^{⊤} K x)^{⊤} = x^{⊤} K^{⊤} x = - x^{⊤} K x

. A number equal to its own negative must be

0

, so only the symmetric part survives — that is why we symmetrize.

The polynomial coefficient of

x_{1} x_{2}

equals the matrix entry

A_{12}

False. The cross term is counted twice (from

A_{12}

and

A_{21}

), so the coefficient equals

A_{12} + A_{21} = 2 A_{12}

. You must halve the coefficient to fill

A

A quadratic form can contain a term like

3 x_{1}

or a constant

+ 7

False. Every term of

x^{⊤} A x

is degree 2 (a product of exactly two components). Linear and constant terms belong to affine/general functions, not to a pure quadratic form.

If all eigenvalues of

A

are positive, then

Q (x) > 0

for all

x \neq = 0

True. In eigen-coordinates

Q = \sum_{i} λ_{i} y_{i}^{2}

; with every

λ_{i} > 0

this is a sum of positive-weighted squares that vanishes only when all

y_{i} = 0

, i.e.

x = 0

A positive semidefinite matrix is automatically positive definite.

False. Semidefinite allows

Q = 0

for some non-zero

x

(a zero eigenvalue / flat direction). Definite is strictly stronger:

Q > 0

except at the origin.

Negating a matrix,

A \mapsto - A

, turns a positive definite form into a negative definite one.

True.

Q (x) = x^{⊤} A x

becomes

- Q (x)

, flipping every eigenvalue's sign, so the upward bowl becomes a downward dome — see the dome panel (atlas top-right).

A negative definite matrix has all negative leading principal minors.

False. For negative definiteness the leading minors alternate in sign, starting negative:

A_{11} < 0

det A > 0

(for

2 \times 2

). Equivalently,

- A

is positive definite. Example:

[- 2 0 0 - 3]

has

A_{11} = - 2 < 0

but

det = 6 > 0

The eigenvectors of

A

point along the directions where the quadratic surface curves most sharply.

True. Eigenvectors are the principal axes; each eigenvalue is the curvature along its axis (largest

λ

= sharpest upward bend, most negative = sharpest downward bend).

A negative semidefinite form (a "ridge") can still take a strictly positive value somewhere.

False. Negative semidefinite means

Q \leq 0

everywhere; along its zero-eigenvalue direction

Q = 0

, and in every other direction

Q < 0

. It never rises above zero — that is exactly the upside-down trough.

Spot the error

Q = 6 x_{1} x_{2} + x_{2}^{2}

, so

A = [0661]

The off-diagonal must be halved:

A_{12} = A_{21} = 6/2 = 3

, giving

A = [0331]

. The uncorrected version would double-count the cross term.

A

has positive diagonal, therefore

A ≻ 0

, so gradient descent has a unique minimum."

Two errors chained. Positive diagonal does not imply

A ≻ 0

(positive definite; off-diagonals can dominate), so the definiteness conclusion is unjustified — check leading minors or eigenvalues first.

Q = x_{1}^{2} - x_{2}^{2}

is negative somewhere and positive somewhere, so it must be positive semidefinite."

Taking both signs means it is indefinite (a saddle), which is the opposite of semidefinite. Semidefinite would require

Q

to never change sign.

Q = - x_{1}^{2} - x_{2}^{2}

is indefinite because everything is negative."

No — every non-zero

x

gives

Q < 0

and the sign never flips, so it is negative definite (a dome with a unique maximum), not indefinite. Indefinite requires both signs to occur.

A = [2152]

gives the form

2 x_{1}^{2} + 6 x_{1} x_{2} + 2 x_{2}^{2}

This

A

is not symmetric, so first symmetrize:

S = [2332]

. The cross coefficient is

S_{12} + S_{21} = 6

— the number happens to match, but the reasoning must go through the symmetric part

(5 + 1) /2 = 3

, not the raw entries.

"Since

det A > 0

, the matrix is positive definite."

For

2 \times 2

det > 0

only says the two eigenvalues share a sign — they could both be negative. Sylvester needs

A_{11} > 0

and

det A > 0

together;

[- 1 0 0 - 2]

has

det = 2 > 0

yet is negative definite.

"Completing the square gave

Q = (x_{1} + x_{2})^{2} - 2 x_{2}^{2}

, and squares are non-negative, so

Q \geq 0

: positive semidefinite."

The second square carries a negative coefficient. Taking

x_{1} = - x_{2}

kills the first square and leaves

- 2 x_{2}^{2} < 0

, so

Q

is actually indefinite. Signs of the square coefficients decide everything.

Why questions

Why do we insist

A

be symmetric when defining a quadratic form?

Because the skew-symmetric part contributes exactly zero to

Q

(a scalar equalling its own negative must be

0

), symmetrizing loses nothing and makes

A

diagonalizable by an orthonormal basis (Spectral Theorem), which is what lets us read off definiteness cleanly.

Why does the cross-term coefficient get halved to build

A

The double sum

\sum_{ij} A_{ij} x_{i} x_{j}

hits the pair

(1, 2)

twice — once as

A_{12} x_{1} x_{2}

and once as

A_{21} x_{2} x_{1}

. With

A

symmetric these add to

2 A_{12}

, so

A_{12}

is half the visible coefficient.

Why is definiteness a property of

A

's eigenvalues rather than of sampled points?

The Spectral theorem rewrites

Q

\sum_{i} λ_{i} y_{i}^{2}

in rotated coordinates, so the sign behaviour over all directions is completely captured by the signs of the

λ_{i}

— samples only probe individual directions.

Why does a positive-definite Hessian guarantee a nice minimum in optimization?

A positive-definite Hessian means the loss curves upward in every direction, so locally it is a convex bowl with a single lowest point — exactly the situation Convex optimization and gradient descent handle well.

Why does completing the square give exactly the

L D L^{⊤}

/ Cholesky factors?

Walk the algebra on

Q = x_{1}^{2} + 2 x_{1} x_{2} + 3 x_{2}^{2}

step by step. Step 1 — collect the $x_{1}$ terms:

x_{1}^{2} + 2 x_{1} x_{2} = (x_{1} + x_{2})^{2} - x_{2}^{2}

, because expanding

(x_{1} + x_{2})^{2}

overshoots by exactly

x_{2}^{2}

, which we subtract back. Step 2 — the shift inside the square is a row of $L$ : the combination inside is

x_{1} + 1 \cdot x_{2}

, and that multiplier

1

is precisely

L_{21}

— it records "how much

x_{2}

to add to

x_{1}

to complete the square". Step 3 — combine leftovers:

Q = (x_{1} + x_{2})^{2} + (3 - 1) x_{2}^{2} = (x_{1} + x_{2})^{2} + 2 x_{2}^{2}

. Step 4 — the coefficient in front of each finished square is a diagonal entry of $D$ : here

1

and

2

, so

D = diag (1, 2)

. Assembling,

L = [1101]

D = [1002]

and indeed

A = L D L^{⊤}

. Cholesky simply folds

D

into

L

, so every algebraic shift you performed is a matrix entry — the procedure is bookkeeping made mechanical.

Why can't a saddle surface have a unique minimum at the origin?

An indefinite form curves up along one eigenvector and down along another, so moving along the downhill axis decreases

Q

below zero — the origin is a stationary point but not a minimum.

Why is

Q (x)

invariant under the rotation

y = P^{⊤} x

from the spectral decomposition?

The Spectral theorem writes

A = P Λ P^{⊤}

where

P

is the orthonormal matrix of eigenvectors and

Λ = diag (λ_{1}, \dots, λ_{n})

holds the eigenvalues. Since

P^{⊤} P = I

, the rotation preserves length, and substituting gives

x^{⊤} A x = x^{⊤} P Λ P^{⊤} x = y^{⊤} Λ y = \sum_{i} λ_{i} y_{i}^{2}

; the same surface is merely viewed along its principal axes, so its shape and definiteness are unchanged.

Edge cases

What is the definiteness of the zero matrix

A = 0

Q (x) = 0

for every

x

, so it is both positive semidefinite and negative semidefinite, but neither strictly definite nor indefinite — a completely flat plane (bottom-right panel of the contour atlas).

A

has one positive eigenvalue and one zero eigenvalue, what shape is

Q

Positive semidefinite: it is a trough (a valley flat along the zero-eigenvalue direction), non-negative everywhere but equal to zero along a whole line, not just the origin — see the trough panel (surface atlas bottom-right).

A

has one negative eigenvalue and one zero eigenvalue, what shape is

Q

Negative semidefinite — a ridge, the mirror image of the trough: flat along the zero-eigenvalue direction and sloping downward everywhere else, so

Q \leq 0

with equality only along that flat line.

For a

1 \times 1

matrix

A = [a]

and scalar

x

, what does

Q

reduce to?

Q = a x^{2}

, the familiar parabola — positive definite iff

a > 0

, showing the quadratic form is literally the matrix generalization of

a x^{2}

What is

Q

at the origin

x = 0

, and what does that tell you about classification?

Q (0) = 0

always, for every matrix, so the origin never distinguishes definiteness types — you must examine non-zero directions.

Can an indefinite matrix have a positive determinant in dimension

n = 2

No. In

2 \times 2

det A = λ_{1} λ_{2}

; indefinite means opposite signs, forcing

det A < 0

. (In higher dimensions a positive determinant can still hide mixed signs, so this shortcut is

2 \times 2

-only.)

What does a negative definite matrix look like as a surface, and how do its minors behave?

A downward dome with a unique maximum at the origin (its contours are still closed ellipses, but height decreases outward). Its leading minors alternate:

A_{11} < 0, det A > 0

for

2 \times 2

— equivalently

- A

is positive definite.

If every leading principal minor of

A

is positive, is

A

positive definite even when

n

is large?

Yes — that is exactly Sylvester's criterion, which extends to all

n

: positive definiteness holds iff all leading minors are positive, not just the determinant.

Is a diagonal matrix with entries

{2, 0, - 3}

definite, semidefinite, or indefinite?

Indefinite, because the eigenvalues (the diagonal entries) contain both a positive

2

and a negative

- 3

; the zero merely adds a flat direction but the mixed signs dominate the classification.

Recall One-line self-test

Cover every answer, run the list top to bottom, and re-read only the ones whose reason you couldn't state cleanly.

Connections

Symmetric matrices — the only matrices these questions assume.
Eigenvalues and eigenvectors — the true arbiter of every definiteness trap here.
Spectral theorem — why sign-of-eigenvalues works.
Positive definite matrices — the "all positive" case.
Convex optimization / Hessian matrix — where the traps bite in ML.
Cholesky decomposition — the completing-the-square edge cases.
PCA — quadratic forms under a length constraint.