Quadratic forms

WHAT is a quadratic form?

WHY symmetric? Only the symmetric part of $A$ affects $Q$. Split $A = S + K$ where $S=\tfrac12(A+A^\top)$ is symmetric and $K=\tfrac12(A-A^\top)$ is skew. Then $\mathbf{x}^\top K \mathbf{x}=0$ always (a scalar equals its transpose: $\mathbf{x}^\top K\mathbf{x}=(\mathbf{x}^\top K\mathbf{x})^\top=\mathbf{x}^\top K^\top\mathbf{x}=-\mathbf{x}^\top K\mathbf{x}$, so it's 0). So we always assume $A$ symmetric — nothing is lost.

HOW to expand it (derivation from scratch)

Start with the double sum and see where each piece comes from. Take $n=2$: $\mathbf{x}^\top A\mathbf{x} = \begin{bmatrix}x_1 & x_2\end{bmatrix}\begin{bmatrix}a & b\\ b & c\end{bmatrix}\begin{bmatrix}x_1\\x_2\end{bmatrix}.$

Step 1 — inner multiply $A\mathbf{x}$: $A\mathbf{x} = \begin{bmatrix}a x_1 + b x_2\\ b x_1 + c x_2\end{bmatrix}.$ Why this step? Matrix–vector product first collapses one dimension, leaving a vector.

Step 2 — dot with $\mathbf{x}^\top$: $Q = x_1(a x_1 + b x_2) + x_2(b x_1 + c x_2) = a x_1^2 + 2b\,x_1 x_2 + c\,x_2^2.$ Why this step? The cross terms $b x_1x_2$ appear twice (from $A_{12}$ and $A_{21}$), producing the factor $2b$. This is the key reading rule.

Definiteness — the shape of the bowl

HOW to test it — via eigenvalues (derivation). Since $A$ is symmetric, the Spectral Theorem gives an orthonormal eigenbasis $A = Q\Lambda Q^\top$ with $Q^\top Q = I$. Substitute $\mathbf{y}=Q^\top\mathbf{x}$ (a rotation, $\|\mathbf y\|=\|\mathbf x\|$): $\mathbf{x}^\top A\mathbf{x} = \mathbf{x}^\top Q\Lambda Q^\top\mathbf{x} = \mathbf{y}^\top\Lambda\mathbf{y} = \sum_{i=1}^n \lambda_i\, y_i^2.$

Worked examples

Common mistakes

Recall flashcards

Connections

• Symmetric matrices — quadratic forms are defined by them.
• Eigenvalues and eigenvectors — decide definiteness and give principal axes.
• Spectral theorem — justifies the $\sum\lambda_i y_i^2$ rewrite.
• Positive definite matrices — the "bowl" case.
• Convex optimization / Hessian matrix — second-order behaviour of loss functions.
• PCA — maximizing $\mathbf x^\top \Sigma\mathbf x$ under $\|\mathbf x\|=1$ is a quadratic form problem.
• Cholesky decomposition — algorithmic completing-the-square.

Concept Map

Hinglish (regional understanding)