First-order optimality conditions — gradient = 0: Worked examples

Prerequisites you should have met: Gradient and directional derivatives (what $\nabla f$ is), Hessian matrix and second-order conditions (the $2 \times 2$ matrix of second derivatives that classifies a point), and Saddle points. Everything else is built here.

Definition Eigenvalue — recap, because the 2-D tests hinge on it

The Hessian $H$ is the $2 \times 2$ table of second derivatives $(f_{xx} f_{x y} f_{x y} f_{y y})$ ; each entry is a curvature. An eigenvalue of $H$ is a number $λ$ such that squeezing the surface along one special direction gives curvature exactly $λ$ — no twisting, pure bending. Picture the surface near a stationary point: there are two perpendicular "principal" directions, and the eigenvalues are how sharply the surface bends up ( $λ > 0$ ) or down ( $λ < 0$ ) along each. So:

both $λ > 0$ → bends up in every direction → minimum (bowl);
both $λ < 0$ → bends down everywhere → maximum (dome);
opposite signs → up one way, down the other → saddle;
one $λ = 0$ → flat in a direction → degenerate, test is silent.

Reading them off is easy in two friendly cases: if $H$ is diagonal, the eigenvalues are just the two diagonal entries; more generally they solve $det (H - λ I) = 0$ , and a quick shortcut is that $det H = λ_{1} λ_{2}$ — so $det H < 0$ forces opposite signs (a saddle), no computation needed.

The scenario matrix

Cell	Case class	Signal at the stationary point	Example
A	Clean minimum (1-D)	$f^{''} > 0$	Ex 1
B	Clean maximum (1-D)	$f^{''} < 0$	Ex 1 (same function)
C	Degenerate 1-D ( $f^{''} = 0$ ) — inflection vs flat min	$f^{''} = 0$ , must probe higher	Ex 2
D	2-D minimum (positive-definite Hessian)	both eigenvalues $> 0$	Ex 3
E	2-D maximum (negative-definite Hessian)	both eigenvalues $< 0$	Ex 4
F	2-D saddle (indefinite Hessian)	eigenvalues opposite sign	Ex 5
G	2-D degenerate (singular Hessian) — a whole valley line	an eigenvalue $= 0$	Ex 6
H	Multiple stationary points, mixed types	classify each separately	Ex 7
I	Boundary / constrained — gradient need NOT vanish	check edge, not just interior	Ex 8
J	Word problem (real quantity to optimise)	model then apply FONC	Ex 9

The four 2-D shapes (cells D–G)

The four 2-D outcomes are exactly the four surface shapes below. Read the figure as a $2 \times 2$ grid, and map each panel to its cell:

Top-left, blue bowl → cell D (minimum): both eigenvalues $> 0$ , every slice curves up; the black dot sits at the lowest point.
Top-right, orange dome → cell E (maximum): both eigenvalues $< 0$ , every slice curves down; the dot is the summit.
Bottom-left, red saddle → cell F (saddle): eigenvalues of opposite sign; the surface rises along one ridge and falls along the perpendicular one, meeting at the dot.
Bottom-right, green valley → cell G (degenerate): one eigenvalue is $0$ , so the trough is flat along a whole line — the dot is only one of infinitely many lowest points.

In every panel the black dot is where $\nabla f = 0$ ; the shape around it is what the eigenvalue signs are secretly describing. Keep this picture in view as you work Ex 3–6.

Figure — First-order optimality conditions — gradient = 0

Cells A & B — the cleanest 1-D case

Worked example Classify every stationary point of

$f (x) = \frac{1}{3} x^{3} - 4 x$ Forecast: guess first — how many stationary points, and is each a max, a min, or neither?

Step 1. Differentiate: $f^{'} (x) = x^{2} - 4$ . Why this step? FONC (the First-Order Necessary Condition) says candidates live where the derivative is zero, so we need $f^{'}$ before anything else.

Step 2. Solve $f^{'} (x) = 0$ : $x^{2} = 4 \Rightarrow x = \pm 2$ . Why? These are the only interior points where "no direction is downhill" can hold.

Step 3. Second derivative $f^{''} (x) = 2 x$ . Why? FONC cannot tell a max from a min (both are flat). The sign of $f^{''}$ measures whether the tangent curves up (bowl) or down (dome).

Step 4. Evaluate:

$f^{''} (2) = 4 > 0$ → curves upward → local minimum (cell A). Value $f (2) = \frac{8}{3} - 8 = - \frac{16}{3}$ .
$f^{''} (- 2) = - 4 < 0$ → curves downward → local maximum (cell B). Value $f (- 2) = - \frac{8}{3} + 8 = \frac{16}{3}$ .

Verify: just left of $x = 2$ , say $x = 1.9$ , $f^{'} (1.9) = 3.61 - 4 = - 0.39 < 0$ (going down); just right, $x = 2.1$ , $f^{'} (2.1) = 4.41 - 4 = 0.41 > 0$ (going up). Down-then-up = a valley = minimum. ✓ Sign flip is opposite at $x = - 2$ . ✓

Cell C — the degenerate 1-D trap ( $f^{''} = 0$ )

Worked example Classify

$x = 0$ for (i) $f (x) = x^{3}$ and (ii) $g (x) = x^{4}$ Forecast: both are flat at $0$ with zero curvature — do they behave the same?

Step 1. $f^{'} (x) = 3 x^{2} \Rightarrow f^{'} (0) = 0$ ; $g^{'} (x) = 4 x^{3} \Rightarrow g^{'} (0) = 0$ . Both are stationary. Why? Confirm we're even in the FONC shortlist.

Step 2. $f^{''} (x) = 6 x \Rightarrow f^{''} (0) = 0$ ; $g^{''} (x) = 12 x^{2} \Rightarrow g^{''} (0) = 0$ . Both second-order tests are silent. Why this step? This is exactly the degenerate cell — we cannot stop here.

Step 3. Probe higher derivatives. First non-zero one:

$f$ : $f^{'''} (0) = 6 \neq = 0$ . An odd-order derivative being first non-zero ⟹ inflection (neither max nor min).
$g$ : $g^{''''} (0) = 24 > 0$ . An even order, positive ⟹ minimum. Why? The lowest surviving term governs the local shape: $f \approx x^{3}$ (odd, changes sign) vs $g \approx x^{4}$ (even, always $\geq 0$ ).

Verify: $f (- 0.1) = - 0.001 < 0 < 0.001 = f (0.1)$ — sign flips, so not an extremum ✓. $g (\pm 0.1) = 0.0001 > 0 = g (0)$ — both sides higher, so a minimum ✓.

Cell D — the 2-D bowl (positive-definite Hessian)

Worked example Minimise

$f (x, y) = x^{2} + 3 y^{2} - 2 x + 6 y + 10$ Forecast: a sum of squares in disguise — one stationary point, and it should be a min. Where?

Step 1. Gradient: $\nabla f = (2 x - 2 6 y + 6)$ . Why? Each entry is $\partial f / \partial x_{i}$ ; FONC needs the whole vector to vanish.

Step 2. Set $\nabla f = 0$ : $2 x - 2 = 0 \Rightarrow x = 1$ ; $6 y + 6 = 0 \Rightarrow y = - 1$ . Why? The unique interior candidate is $(1, - 1)$ .

Step 3. Hessian $H = (2006)$ . Why this tool? In 2-D the sign of one number ( $f^{''}$ ) becomes a matrix of curvatures; its definiteness tells us the shape.

Step 4. Diagonal Hessian, so the eigenvalues are just the diagonal entries $2$ and $6$ , both $> 0$ ⟹ positive definite ⟹ curves up in every direction ⟹ local (here global) minimum (cell D). This is the upward bowl in the top-left of the figure above. Value $f (1, - 1) = 1 + 3 - 2 - 6 + 10 = 6$ .

Verify (complete the square): $f = (x - 1)^{2} + 3 (y + 1)^{2} + 6$ , a sum of squares $\geq 6$ , equal to $6$ only at $(1, - 1)$ . ✓ Matches.

Cell E — the 2-D dome (negative-definite Hessian)

Worked example Find and classify the stationary point of

$f (x, y) = - x^{2} - 2 y^{2} + 4 x + 4 y + 1$ Forecast: every squared term is negative — a downward dome. Expect a single maximum.

Step 1. Gradient: $\nabla f = (- 2 x + 4 - 4 y + 4)$ . Why? FONC needs the full gradient before anything.

Step 2. Set $\nabla f = 0$ : $- 2 x + 4 = 0 \Rightarrow x = 2$ ; $- 4 y + 4 = 0 \Rightarrow y = 1$ . Unique candidate $(2, 1)$ . Why? Linear equations, one solution.

Step 3. Hessian $H = (- 2 0 0 - 4)$ . Why this tool? The 2-D analogue of " $f^{''} < 0$ ⟹ maximum". We now need the whole matrix to be negative definite.

Step 4. Diagonal, so eigenvalues are $- 2$ and $- 4$ , both $< 0$ ⟹ negative definite ⟹ curves down in every direction ⟹ local maximum (cell E). This is the downward dome in the top-right of the figure above. Value $f (2, 1) = - 4 - 2 + 8 + 4 + 1 = 7$ . Why the test differs from cell D: two negative eigenvalues means every direction is downhill from the point — the exact mirror image of the bowl.

Verify (complete the square): $f = - (x - 2)^{2} - 2 (y - 1)^{2} + 7$ . Two negative squares means $f \leq 7$ , with equality only at $(2, 1)$ . ✓ A genuine maximum of value $7$ .

Cell F — the 2-D saddle (indefinite Hessian)

Worked example Classify the stationary point of

$f (x, y) = x^{2} - 4 x y + y^{2}$ Forecast: the cross term $- 4 x y$ is large — sniff a saddle.

Step 1. $\nabla f = (2 x - 4 y - 4 x + 2 y) = 0$ . Why? FONC.

Step 2. Solve: $2 x - 4 y = 0 \Rightarrow x = 2 y$ ; substitute into $- 4 (2 y) + 2 y = - 6 y = 0 \Rightarrow y = 0$ , so $x = 0$ . Only stationary point: $(0, 0)$ . Why? Linear system, unique solution.

Step 3. Hessian $H = (2 - 4 - 4 2)$ . Its determinant $det H = 2 \cdot 2 - (- 4) (- 4) = 4 - 16 = - 12 < 0$ . Why this tool? A negative determinant of a $2 \times 2$ Hessian forces eigenvalues of opposite sign (recall $det H = λ_{1} λ_{2}$ ) — the definitive saddle signature.

Step 4. Eigenvalues solve $(2 - λ)^{2} - 16 = 0 \Rightarrow λ = 2 \pm 4 = 6, - 2$ . Opposite signs ⟹ saddle (cell F): up along one axis, down along the other. This is the saddle shape in the bottom-left of the figure above.

Verify: along the line $y = x$ : $f = x^{2} - 4 x^{2} + x^{2} = - 2 x^{2} < 0$ (falls). Along $y = - x$ : $f = x^{2} + 4 x^{2} + x^{2} = 6 x^{2} > 0$ (rises). Both behaviours through $(0, 0)$ = saddle. ✓ Eigenvalues $6, - 2$ match these two directions.

Cell G — the degenerate 2-D valley (singular Hessian)

Worked example Classify the stationary points of

$f (x, y) = (x - y)^{2}$ Forecast: it's a square, so $f \geq 0$ — but where is it exactly zero, and how many "minima" are there?

Step 1. $\nabla f = (2 (x - y) - 2 (x - y)) = 0$ . Why? FONC.

Step 2. Both equations reduce to the same condition $x - y = 0$ , i.e. $x = y$ . So the stationary set is the whole line $y = x$ , not a single point (cell G). Why this matters: FONC can hand you a curve of candidates, not just isolated dots.

Step 3. Hessian $H = (2 - 2 - 2 2)$ , $det H = 4 - 4 = 0$ ⟹ singular ⟹ an eigenvalue is $0$ ⟹ Hessian test is inconclusive. Why? Zero determinant means one direction has no curvature — a flat trough — so the second-order matrix alone can't finish the job. This is the flat-bottomed valley in the bottom-right of the figure above.

Step 4. Argue directly: $f = (x - y)^{2} \geq 0$ everywhere, and equals $0$ exactly on $y = x$ . So every point of that line is a (non-strict, global) minimum — a flat-bottomed valley (cell G).

Verify: eigenvalues of $H$ are $4$ and $0$ (trace $4$ , det $0$ ). The $0$ -eigenvalue direction is along $(1, 1)$ — indeed moving along $y = x$ keeps $f = 0$ constant. ✓

Cell H — multiple stationary points of mixed type

Worked example Find and classify ALL stationary points of

$f (x, y) = x^{3} - 3 x + y^{2}$ Forecast: cubic in $x$ hints at two $x$ -candidates; parabola in $y$ is a simple min. Expect one min and one saddle.

Step 1. $\nabla f = (3 x^{2} - 3 2 y)$ . Why? FONC needs the full gradient.

Step 2. Set $= 0$ : $2 y = 0 \Rightarrow y = 0$ ; $3 x^{2} - 3 = 0 \Rightarrow x = \pm 1$ . Two stationary points: $(1, 0)$ and $(- 1, 0)$ . Why? Solve component-wise; the $y$ -equation is easy, the $x$ -equation gives two roots.

Step 3. Hessian $H = (6 x 0 02)$ . Why? Classify each point separately — the entry $6 x$ changes sign between them.

Step 4. Evaluate (diagonal ⟹ eigenvalues are the diagonal entries):

At $(1, 0)$ : $H = (6002)$ , eigenvalues $6, 2 > 0$ ⟹ local minimum (bowl, cell D-type). Value $f (1, 0) = 1 - 3 + 0 = - 2$ .
At $(- 1, 0)$ : $H = (- 6 0 02)$ , eigenvalues $- 6, 2$ opposite signs ⟹ saddle (cell F-type). Value $f (- 1, 0) = - 1 + 3 + 0 = 2$ .

Verify: near $(1, 0)$ , $f (1 + h, 0) = (1 + h)^{3} - 3 (1 + h) = - 2 + 3 h^{2} + h^{3} \approx - 2 + 3 h^{2} > - 2$ (rises) ✓ min. Near $(- 1, 0)$ along $x$ : $f (- 1 + h, 0) = 2 - 3 h^{2} + \dots < 2$ (falls); along $y$ : $f (- 1, k) = 2 + k^{2} > 2$ (rises) — up and down = saddle ✓.

Cell I — the boundary case (gradient does NOT vanish at the optimum)

Worked example On the closed disk

${x^{2} + y^{2} \leq 4}$ , find the min and max of $f (x, y) = x^{2} + y^{2}$ Forecast: the farthest-out points from the origin win the max — will $\nabla f = 0$ find them?

Step 1. Interior candidates: $\nabla f = (2 x, 2 y) = 0 \Rightarrow (0, 0)$ , giving $f = 0$ — the minimum, and clearly not the maximum. Why? FONC only shortlists interior flat points.

Step 2. For the maximum, check the boundary circle $x^{2} + y^{2} = 4$ . There $f = x^{2} + y^{2} = 4$ everywhere on the edge. Why this step? The derivation of FONC does not apply on the boundary, so we evaluate $f$ there directly.

Step 3. Compare: interior gives $f = 0$ ; boundary gives $f = 4$ . Maximum value $= 4$ , attained on the whole circle — at points where $\nabla f = (2 x, 2 y)$ has magnitude $(2 x)^{2} + (2 y)^{2} = 2 \cdot 2 = 4 \neq = 0$ (it points radially outward). Why it matters: the optimum has a non-zero gradient — FONC alone would have missed it entirely (cell I).

Verify: at $(2, 0)$ , $f = 4$ and $\nabla f = (4, 0) \neq = 0$ , pointing straight out of the disk — you'd love to keep going but the boundary stops you. ✓

Cell J — a real-world word problem

Worked example A rectangular open-top box (no lid) must hold volume

$32 cm^{3}$ . Minimise its surface area. Forecast: guess whether the base is square and whether the box is tall or squat.

Step 1 (model). Let base be $x \times y$ , height $z$ , all in cm. Volume: $x y z = 32 \Rightarrow z = \frac{32}{x y}$ . Why? Reduce to two free variables so we can apply the 2-D FONC.

Step 2 (objective). Open top surface = base + 4 walls: $S (x, y) = x y + 2 x z + 2 y z = x y + 2 x \cdot \frac{32}{x y} + 2 y \cdot \frac{32}{x y} = x y + \frac{64}{y} + \frac{64}{x} .$ Why? Substitute the constraint so $S$ depends only on $x, y$ — now it's unconstrained and FONC applies.

Step 3 (FONC). Set the gradient to zero: $\frac{\partial S}{\partial x} = y - \frac{64}{x ^{2}} = 0, \frac{\partial S}{\partial y} = x - \frac{64}{y ^{2}} = 0.$ Why this step? At an interior minimum the surface is flat in both directions — that is exactly $\nabla S = 0$ .

Step 4 (solve). From the first equation, $y = \frac{64}{x ^{2}}$ . Substitute into the second: $x = \frac{64}{y ^{2}} = \frac{64}{( 64/ x ^{2} ) ^{2}} = 64 \cdot \frac{x ^{4}}{6 4 ^{2}} = \frac{x ^{4}}{64} \Rightarrow x^{3} = 64 \Rightarrow x = 4,$ then $y = 64/16 = 4$ and $z = 32/ (4 \cdot 4) = 2$ . Why? We only keep the physically sensible root $x, y > 0$ ; it gives one candidate box.

Step 5 (classify). Second derivatives at $(4, 4)$ : $S_{xx} = \frac{128}{x ^{3}} = 2$ , $S_{y y} = \frac{128}{y ^{3}} = 2$ , and $S_{x y} = 1$ . So $H = (2112), det H = 2 \cdot 2 - 1 \cdot 1 = 3 > 0, S_{xx} = 2 > 0.$ Why this step? FONC only shortlists; a positive determinant with a positive top-left entry means both eigenvalues are positive ( $det H = λ_{1} λ_{2} > 0$ and their sum $= 4 > 0$ forces both $> 0$ ) ⟹ positive definite ⟹ genuine minimum, not a saddle.

Verify (units + value): $x = y = 4 cm, z = 2 cm$ ; volume $= 4 \cdot 4 \cdot 2 = 32 cm^{3}$ ✓ (matches the constraint). Surface $S = 16 + 16 + 16 = 48 cm^{2}$ . Note the base is square and the box is half as tall as it is wide — the classic open-box answer. ✓

Flashcards

When

f^{''} (x^{*}) = 0

, what do you do?

The second-order test is silent; probe the first non-zero higher derivative (odd order ⇒ inflection, even & positive ⇒ min).

What is an eigenvalue of the Hessian, intuitively?

The pure curvature of the surface along one of its two principal directions; its sign says bends-up (+) or bends-down (−).

2-D Hessian test with

det H > 0

and

f_{xx} > 0

Positive definite ⇒ local minimum.

2-D Hessian test with

det H > 0

and

f_{xx} < 0

Negative definite ⇒ local maximum.

2-D Hessian test with

det H < 0

Indefinite ⇒ saddle (eigenvalues of opposite sign, since

det H = λ_{1} λ_{2} < 0

2-D Hessian test with

det H = 0

Inconclusive (singular) — a flat direction; classify by other means.

Where can an optimum hide even though

\nabla f \neq = 0

there?

On the boundary of a closed domain — FONC only governs interior points.

Open-top box of volume

32 cm^{3}

minimum surface dimensions?

4 \times 4 \times 2

cm, surface

48 cm^{2}

Recall One-line summary of the whole matrix

Find where $\nabla f = 0$ (maybe zero, one, several, or a whole line of points), then let the Hessian's eigenvalue signs decide min/max/saddle — and if the Hessian is singular or you're on a boundary, argue by hand or use constraints.

Connections

Parent: FONC — the condition every example applies.
Hessian matrix and second-order conditions — the classifier used in cells C–J.
Saddle points — cell F made concrete.
Convex functions — when a single stationary point (cell D) is automatically global.
Lagrange multipliers — the proper tool for the boundary cell I.
Gradient descent — an algorithm that would settle into the minima but get stuck at saddles.

Worked examples — First-order optimality conditions — gradient = 0

The scenario matrix

The four 2-D shapes (cells D–G)

Cells A & B — the cleanest 1-D case

Cell C — the degenerate 1-D trap ( $f^{''} = 0$ )

Cell D — the 2-D bowl (positive-definite Hessian)

Cell E — the 2-D dome (negative-definite Hessian)

Cell F — the 2-D saddle (indefinite Hessian)

Cell G — the degenerate 2-D valley (singular Hessian)

Cell H — multiple stationary points of mixed type

Cell I — the boundary case (gradient does NOT vanish at the optimum)

Cell J — a real-world word problem

Flashcards

Connections

Concept Map