The Hessian matrix: Worked examples — AI-ML deep dive

Before anything, one reminder in plain words:

The scenario matrix

Every $2 \times 2$ critical point falls into exactly one cell below. The " $D$ , $a$ , eigenvalues" columns are the fingerprint you read off; the "shape" is the verdict. Here $a = f_{xx}$ is the top-left diagonal curvature (not to be confused with $c = f_{y y}$ or with the eigenvalues). The three $D = 0$ cases (E, F, I) are all inconclusive but have distinct eigenvector structure, so we split them out.

Cell	Situation	$D = det H$	$a = f_{xx}$	eigenvalues (with structure)	Shape / verdict
A	pure bowl (no coupling)	$D > 0$	$a > 0$	$+, +$	local minimum
B	pure dome	$D > 0$	$a < 0$	$-, -$	local maximum
C	saddle	$D < 0$	any	$+, -$	saddle
D	tilted valley (off-diagonal $b \neq = 0$ )	$D > 0$	$a > 0$	$+, +$ , eigenvectors rotated	minimum, rotated axes
E	one flat direction, $b = 0$ (axis-aligned)	$D = 0$	—	one $+$ , one $0$ ; flat axis = $x$ or $y$	inconclusive, check higher order
F	zero Hessian, $H = 0$	$D = 0$	$a = 0$	$0, 0$ ; every direction flat	inconclusive, no eigen-info at all
G	word problem (real units)	—	—	apply the test to a model	classify a real minimum
H	exam twist: gradient $\neq = 0$	—	—	test not licensed here	cannot classify (not critical)
I	one flat direction, $b \neq = 0$ (skewed)	$D = 0$	—	one $+$ , one $0$ ; flat axis is tilted	inconclusive, flat along a tilted line

How the three $D = 0$ cells differ: in E the zero-eigenvalue direction lies along a coordinate axis (because $b = 0$ ), so you can read the flat direction straight off the diagonal. In I the coupling $b \neq = 0$ rotates that flat direction onto a tilted line — you must compute the eigenvector to find it. In F both eigenvalues are zero, so there is no preferred direction at all and the Hessian gives you nothing; only the raw function values decide the shape.

We now hit each cell in the order the matrix lists them: A, B, C, D, E, F, G, H, I.

Cell A — the clean bowl

Worked example

$f (x, y) = 2 x^{2} + 5 y^{2}$ Forecast: two positive coefficients, no $x y$ term. Guess the shape before reading on.

Step 1 — Gradient. $\nabla f = (4 x, 10 y)$ . Set both to zero: $x = 0, y = 0$ , so the only critical point is $(0, 0)$ . Why this step? The test is only meaningful where the ground is flat; we must find that spot first.

Step 2 — Hessian. $f_{xx} = 4, f_{y y} = 10, f_{x y} = 0$ , so $H = [40010] .$ Why this step? $H$ is constant here (a pure quadratic), so the curvature is the same everywhere — the local picture is the global picture.

Step 3 — Read $D$ and $a$ . $D = (4) (10) - 0^{2} = 40 > 0$ and $a = 4 > 0$ . Why this step? $D > 0$ says the two curvatures share a sign; $a > 0$ says that shared sign is positive → curves up in every direction.

Verify: eigenvalues of a diagonal matrix are its diagonal: ${4, 10}$ — both positive, confirming a minimum. Plug $(0.1, 0)$ : $f = 0.02 > 0 = f (0, 0)$ ; plug $(0, 0.1)$ : $f = 0.05 > 0$ . Every nudge raises $f$ . ✅

Figure 1 (below): the surface $f = 2 x^{2} + 5 y^{2}$ rendered as a 3-D bowl, with the plum dot marking the minimum at $(0, 0)$ . Notice the surface rises no matter which way you step off the dot — the picture of "both eigenvalues positive". Look at how the teal mesh curves upward along both the $x$ and $y$ ridges.

Cell B — the dome

Cell C — the saddle

Figure 2 (below): the saddle surface $f = 3 x^{2} - 2 y^{2}$ . Follow the teal ridge along $x$ (curving up) and the orange ridge along $y$ (curving down); the plum dot at $(0, 0)$ is simultaneously a valley and a peak depending on which way you walk — the geometric meaning of mixed-sign eigenvalues.

Cell D — the tilted valley (off-diagonal bites)

This is the case people fumble: the surface is still a bowl, but its axes are rotated, so you cannot eyeball it from the diagonal alone.

Worked example

$f (x, y) = x^{2} + x y + y^{2}$ Forecast: the $x y$ term couples the directions. Bowl, dome, or saddle — and are the "natural axes" $x$ and $y$ ?

Step 1 — Gradient. $\nabla f = (2 x + y, x + 2 y)$ . Set to zero: from the first, $y = - 2 x$ ; substitute into the second, $x + 2 (- 2 x) = - 3 x = 0 \Rightarrow x = 0$ , hence $y = 0$ . Critical point $(0, 0)$ . Why this step? With coupling, the two equations must be solved together; you cannot zero them independently.

Step 2 — Hessian. $f_{xx} = 2, f_{y y} = 2, f_{x y} = 1$ : $H = [2112] .$ Why this step? The off-diagonal $b = 1 \neq = 0$ records that the $x$ -slope changes as you move in $y$ — the coupling the parent note called "twist".

Step 3 — Read $D$ and $a$ . $D = (2) (2) - 1^{2} = 3 > 0$ , $a = 2 > 0$ → minimum. Why this step? Same fingerprint as Cell A ( $D > 0$ , $a > 0$ ), so the verdict is a minimum — the coupling did not change what it is, only its orientation.

Step 4 — Eigenvalues, because we want the real axes. Solve $det (H - λ I) = 0$ : $(2 - λ)^{2} - 1 = 0 \Rightarrow 2 - λ = \pm 1 \Rightarrow λ = 1, 3$ . Why this step? The eigenvalues $1$ and $3$ are the true curvatures, and their eigenvectors $(1, - 1)$ and $(1, 1)$ are the valley's real axes — tilted $4 5^{\circ}$ from $x, y$ . The det/trace shortcut confirms the verdict; the eigen-decomposition reveals the geometry. See Positive definite matrices for why "all eigenvalues $> 0$ " is exactly the convexity condition.

Verify: product of eigenvalues $1 \cdot 3 = 3 = D$ ✅; sum $1 + 3 = 4 = tr H = 2 + 2$ ✅. Both positive → minimum, consistent with the shortcut.

Figure 3 (below): the contour map of $f = x^{2} + x y + y^{2}$ . The teal ellipses are the level curves; the orange arrow is the steep eigen-axis $(1, 1)$ with $λ = 3$ , and the plum arrow is the gentle eigen-axis $(1, - 1)$ with $λ = 1$ . Notice the ellipses are squashed along the orange arrow — big curvature there — and stretched along the plum arrow. The bowl's real axes are tilted $4 5^{\circ}$ , exactly what the eigenvectors report.

Cell E — a genuine flat direction, no coupling (inconclusive)

The zero direction here is axis-aligned because $b = 0$ — you can spot the flat axis directly on the diagonal.

Worked example

$f (x, y) = x^{2} + y^{4}$ Forecast: $x^{2}$ is a clear bowl in $x$ , but $y^{4}$ is flatter than a parabola at the origin. What does the Hessian say?

Step 1 — Gradient. $\nabla f = (2 x, 4 y^{3})$ → $(0, 0)$ . Why this step? Both partials vanish only at the origin, so that is the one critical point where the test could possibly apply.

Step 2 — Hessian at the origin. $f_{xx} = 2, f_{y y} = 12 y^{2}, f_{x y} = 0$ . At $(0, 0)$ , $f_{y y} = 0$ : $H = [2000] .$ Why this step? Unlike the pure quadratics above, this $H$ depends on the point; we must evaluate it at the critical point.

Step 3 — Read $D$ , and note the eigen-structure. $D = (2) (0) - 0 = 0$ . Because $b = 0$ , the eigenvalues are just the diagonal ${2, 0}$ , and the zero-eigenvalue direction is the $y$ -axis $(0, 1)$ — no rotation. Why this step? $D = 0$ means one eigenvalue is $0$ — a direction with no second-order curvature. The quadratic Taylor term is blind there, so the test is inconclusive. But the flat direction is easy to name here: it's the $y$ -axis.

Step 4 — Rescue with higher order. Along $y$ (where $x = 0$ ), $f = y^{4} > 0$ for $y \neq = 0$ ; along $x$ , $f = x^{2} > 0$ . So it truly is a minimum — but only the fourth-order term proved it. Why this step? When the Hessian is silent, the surrounding values themselves are the only remaining evidence, so we inspect the function directly along each axis.

Verify: eigenvalues ${2, 0}$ — the zero confirms inconclusiveness of the test, even though the point is a minimum. This is why " $D = 0$ " and "not a minimum" are not the same. ✅

Cell F — the totally flat critical point (zero Hessian)

Both eigenvalues are zero, so — unlike E and I — there is no preferred direction at all; the Hessian tells you nothing.

Worked example

$f (x, y) = x^{3} + y^{3}$ Forecast: cubes. The origin is critical — but is it min, max, saddle?

Step 1 — Gradient. $\nabla f = (3 x^{2}, 3 y^{2})$ → zero only at $(0, 0)$ . Why this step? A square is zero only when its base is zero, so both partials vanish solely at the origin — the lone candidate.

Step 2 — Hessian at origin. $f_{xx} = 6 x, f_{y y} = 6 y, f_{x y} = 0$ . At $(0, 0)$ every entry is $0$ : $H = [0000] .$ Why this step? Because $H$ varies with position, we must evaluate it at the critical point; here it collapses to the zero matrix, meaning no second-order curvature in any direction.

Step 3 — Read $D$ and the eigen-structure. $D = 0$ , $a = 0$ ; both eigenvalues are $0$ , so every direction is flat. This is strictly more degenerate than E (which kept one positive curvature): here the Hessian gives zero usable information. Why this step? A zero Hessian gives no curvature information at all — the surface is flat to second order in every direction.

Step 4 — Direct inspection. Along the $x$ -axis $f = x^{3}$ : negative for $x < 0$ , positive for $x > 0$ . So the value goes below and above $f (0, 0) = 0$ near the origin → it is neither a min, max, nor a clean saddle; it's an inflection-type point. Why this step? With the Hessian silent, only the actual function values near the point can settle the verdict, so we walk along an axis and watch the sign of $f$ .

Verify: $f (- 0.1, 0) = - 0.001 < 0$ and $f (0.1, 0) = 0.001 > 0$ . Straddling zero means no local extremum. ✅

Cell G — a word problem with real units

Worked example Minimizing manufacturing cost

A workshop's daily cost (in ₹ thousands) to produce $x$ chairs and $y$ tables is $C (x, y) = x^{2} - 2 x y + 3 y^{2} - 8 x - 14 y + 60.$ Find the production mix that minimizes cost and confirm it is a minimum.

Forecast: there's an $x y$ coupling and negative linear terms — the flat point won't be at the origin.

Step 1 — Gradient = 0. $C_{x} = 2 x - 2 y - 8 = 0, C_{y} = - 2 x + 6 y - 14 = 0.$ Why this step? The lowest cost sits where marginal cost in both goods is zero.

Step 2 — Solve the system. Add the two equations: $(2 x - 2 y - 8) + (- 2 x + 6 y - 14) = 4 y - 22 = 0 \Rightarrow y = 5.5$ . Then $2 x - 2 (5.5) - 8 = 0 \Rightarrow 2 x = 19 \Rightarrow x = 9.5$ . Critical point $(9.5, 5.5)$ . Why this step? Coupled linear equations solve together — eliminate one variable first.

Step 3 — Hessian. $C_{xx} = 2, C_{y y} = 6, C_{x y} = - 2$ : $H = [2 - 2 - 2 6] .$ Why this step? Linear terms ( $- 8 x - 14 y$ ) vanish under the second derivative — they set where the minimum is, not whether it's a minimum.

Step 4 — Classify. $D = (2) (6) - (- 2)^{2} = 12 - 4 = 8 > 0$ and $a = 2 > 0$ → minimum. Why this step? This is the Cell A fingerprint applied to a real model: $D > 0$ with positive top-left curvature guarantees the cost bowl opens upward, so the flat point really is the cheapest.

Conclusion (with units): the cheapest production mix is $x = 9.5$ chairs and $y = 5.5$ tables per day, giving minimum cost $C (9.5, 5.5) = - 16.5$ (₹ thousand) — i.e. the model's baseline offset makes the optimized daily cost read as ₹ $- 16, 500$ relative to its constant term; what matters physically is that no nearby mix is cheaper.

Verify (units + value): $C (9.5, 5.5) = 90.25 - 104.5 + 90.75 - 76 - 77 + 60 = - 16.5$ (₹ thousand). Bump to $x = 10$ : $C (10, 5.5) = 100 - 110 + 90.75 - 80 - 77 + 60 = - 16.25 > - 16.5$ ; bump $y = 6$ : $C (9.5, 6) = 90.25 - 114 + 108 - 76 - 84 + 60 = - 15.75 > - 16.5$ . Both directions cost more, confirming the minimum. Units stay ₹ thousand throughout. ✅

Cell H — the exam twist: the gradient is NOT zero

This is the trap that catches careful students: a perfectly nice, positive-definite Hessian at a point that isn't critical. The correct move is to refuse to classify.

Worked example Classify

$f (x, y) = x^{2} + y^{2}$ at the point $(1, 1)$ . Forecast: the Hessian is a textbook-looking positive-definite matrix here. Does that make $(1, 1)$ a minimum?

Step 1 — Check the gradient FIRST. $\nabla f = (2 x, 2 y)$ ; at $(1, 1)$ this is $(2, 2) \neq = (0, 0)$ . Why this step? The second-derivative test is only licensed at critical points. Off them, the linear term $\nabla f^{⊤} Δ x$ dominates the Taylor expansion and there is simply no local extremum to classify.

Step 2 — Do NOT proceed to the Hessian test. Even though $H = [2002]$ looks like a textbook minimum, the correct answer is: $(1, 1)$ is not a critical point, so the test does not apply. Why this step? A tempting-looking $H$ is a trap here; refusing to run the test is itself the correct action, because a nonzero gradient means the point is on a slope, not at a bottom.

Step 3 — Where IS the minimum? Setting $\nabla f = (2 x, 2 y) = (0, 0)$ gives $(0, 0)$ — that is the critical point, and there $H = [2002]$ (Cell A) correctly reports a minimum. The point $(1, 1)$ was simply on the slope of the bowl, not at its bottom. Why this step? It shows the same $H$ is only meaningful once you stand at the flat spot.

Verify: move downhill along $- \nabla f = (- 2, - 2)$ : $f (0.9, 0.9) = 1.62 < 2 = f (1, 1)$ . The value drops, so $(1, 1)$ is clearly not a minimum. The gradient check saved us. ✅

Cell I — a skewed flat direction, with coupling (inconclusive)

Cell E's flat direction lay along the $y$ -axis. But a zero eigenvalue can point along a tilted line when the off-diagonal $b \neq = 0$ . Here is that case, so you recognise it too.

Worked example

$f (x, y) = (x - y)^{2} = x^{2} - 2 x y + y^{2}$ Forecast: this is a squared quantity, so $f \geq 0$ everywhere. But along the line $x = y$ it is exactly zero — a whole flat trough. What does the Hessian say, and which direction is flat?

Step 1 — Gradient. $\nabla f = (2 x - 2 y, - 2 x + 2 y)$ . Both are zero whenever $x = y$ — so the critical set is the entire line $x = y$ , not an isolated point. Why this step? We must first find where the slope dies; here it dies along a whole line, our first clue that a flat direction exists.

Step 2 — Hessian. $f_{xx} = 2, f_{y y} = 2, f_{x y} = - 2$ : $H = [2 - 2 - 2 2] .$ Why this step? The off-diagonal $b = - 2 \neq = 0$ signals coupling, so any flat direction will be tilted, not axis-aligned — the key contrast with Cell E.

Step 3 — Read $D$ . $D = (2) (2) - (- 2)^{2} = 4 - 4 = 0$ . Why this step? $D = 0$ again means one eigenvalue is zero — the test is inconclusive by the same reasoning as Cell E.

Step 4 — Find the flat direction via eigenvalues. Solve $(2 - λ)^{2} - 4 = 0 \Rightarrow 2 - λ = \pm 2 \Rightarrow λ = 0, 4$ . The eigenvector for $λ = 0$ satisfies $2 u_{1} - 2 u_{2} = 0 \Rightarrow u_{1} = u_{2}$ : direction $(1, 1)$ . The eigenvector for $λ = 4$ is $(1, - 1)$ . Why this step? The zero-eigenvalue direction $(1, 1)$ is exactly the line $x = y$ where $f$ stays flat; the $λ = 4$ direction $(1, - 1)$ is where the trough walls curve up. Unlike E (flat along an axis), here you must compute the eigenvector to locate the tilted flat line.

Verify: product $0 \cdot 4 = 0 = D$ ✅. Along $(1, 1)$ from the origin: $f (t, t) = (t - t)^{2} = 0$ — perfectly flat, confirming the zero eigenvalue's direction. Along $(1, - 1)$ : $f (t, - t) = (2 t)^{2} = 4 t^{2} > 0$ — curves up, matching $λ = 4$ . ✅

Where each case sends you next

If you found…	It means	Relevant tool
all $λ > 0$ (Cells A, D, G)	local minimum, [[Convex functions	convex]] locally
mixed signs (Cell C)	saddle	Gradient descent can stall / escape slowly
some $λ = 0$ (Cells E, F, I)	flat direction	need higher-order terms
$\nabla f \neq = 0$ (Cell H)	not critical	keep descending

Recall Quick self-test

Cell for $H = [- 3 0 0 - 5]$ at a critical point? ::: Cell B — $D = 15 > 0$ , $a = - 3 < 0$ → maximum. Cell for $H = [1221]$ ? ::: Cell C — $D = 1 - 4 = - 3 < 0$ → saddle (eigenvalues $3, - 1$ ). Cell for $f = x^{2} + y^{4}$ at origin? ::: Cell E — $D = 0$ , flat along the $y$ -axis, test inconclusive (though it's really a min). Cell for $f = (x - y)^{2}$ along $x = y$ ? ::: Cell I — $D = 0$ , flat along the tilted line $(1, 1)$ , test inconclusive. Cell for $f = x^{3} + y^{3}$ at origin? ::: Cell F — $H = 0$ , both eigenvalues zero, fully inconclusive (it's an inflection). Can you classify $x^{2} + y^{2}$ at $(2, 0)$ ? ::: No — Cell H, gradient $(4, 0) \neq = 0$ , not a critical point.