First-order optimality conditions — gradient = 0: Question bank

Reminder of the symbols we lean on (all defined in the parent note, restated here so this page stands alone):

$\nabla f$ — the gradient, the vector of all partial derivatives; it points toward steepest increase. See Gradient and directional derivatives.
A stationary point is any point where $\nabla f = 0$ .
FONC = First-Order Necessary Condition: at an interior optimum, $\nabla f = 0$ .
The Hessian $H$ is the matrix of second partials; it does the classifying — see Hessian matrix and second-order conditions.

Figure 1 — the ±h trapping argument (why interior matters). At an interior minimum you can step both left and right; the right slope is $\geq 0$ , the left slope is $\leq 0$ , and differentiability squeezes both to $0$ .

Figure — First-order optimality conditions — gradient = 0

Figure 2 — the three flat shapes. Minimum (bowl), maximum (dome) and saddle all have $\nabla f = 0$ ; only curvature tells them apart.

Figure 3 — Hessian eigenvalue sign chart. How the signs of the two eigenvalues map to min / max / saddle / inconclusive.

Figure 4 — boundary optimum. On a closed domain the lowest point can sit on the wall, where $\nabla f$ still points outward (non-zero) because you're forbidden to step out.

True or false — justify

Every local minimum in the interior of the domain has

\nabla f = 0

True — this is exactly FONC; if any component of

\nabla f

were non-zero you could step in that coordinate and lower

f

, contradicting "minimum" (Figure 1).

Every point with

\nabla f = 0

is a local minimum.

False — flatness also occurs at maxima and saddles (Figure 2); FONC only shortlists candidates, it does not crown a winner.

\nabla f \neq = 0

x

, then

x

cannot be a local max or min.

True — a non-zero gradient gives a strict downhill direction

u = - \nabla f / ∥ \nabla f ∥

(here

∥ \cdot ∥

is the Euclidean length) and a strict uphill direction

- u

, so you can both increase and decrease

f

; neither extremum is possible.

f (x) = x^{3}

has a minimum at

x = 0

because

f^{'} (0) = 0

False —

f^{'} (0) = 0

makes

0

stationary, but

f

rises to the right and falls to the left, so

0

is an inflection, neither max nor min.

A minimum on a closed domain must satisfy

\nabla f = 0

False — if the minimum sits on the boundary, the gradient can point outward and stay non-zero (Figure 4); you can't step outside the domain, so FONC's "both directions" argument breaks. Use Lagrange multipliers/KKT there.

H

(Hessian) is positive definite at a stationary point, the point is a strict local minimum.

True — positive definite means

f

curves upward in every direction (top-right cell of Figure 3), so nearby

f

-values all exceed

f (x^{*})

; this is the second-order sufficient condition.

Positive definite Hessian at a stationary point guarantees a global minimum.

False in general — it guarantees only a local minimum; globality needs extra structure such as convexity of

f

over the whole domain.

For a convex

f

, every stationary point is a global minimum.

True — convexity means the graph never dips below any tangent plane; since a stationary tangent plane is flat, no point can lie below

x^{*}

, making it global.

The directional derivative

D_{u} f (x^{*}) = \nabla f (x^{*}) \cdot u

is zero in every direction at a stationary point.

True — if

\nabla f (x^{*}) = 0

then

0 \cdot u = 0

for all unit

u

; that "no slope in any direction" is precisely what flatness means.

A saddle point has a non-zero gradient because the surface is clearly tilted.

False — at a saddle

\nabla f = 0

; the surface rises along one axis and falls along another (Figure 2, right), but at the exact centre every first-order slope cancels to zero.

Spot the error

f^{'} (x^{*}) \geq 0

from the right and

f^{'} (x^{*}) \leq 0

from the left, so

f^{'} (x^{*})

could be any number in between." — where's the flaw?

Differentiability forces the left and right limits to be the same number (Figure 1); a single value that is both

\leq 0

and

\geq 0

can only be

0

. There is no "in between".

"We found

\nabla f = 0

, so we're done — it's the minimum we wanted." — what's missing?

The classification step;

\nabla f = 0

is necessary but not sufficient, so you must run the Hessian test (Figure 3) before claiming it's a minimum.

"The gradient points downhill toward the minimum, so gradient descent moves along

+ \nabla f

." — fix it.

The gradient points toward steepest increase; to descend you move along

- \nabla f

. See Gradient descent.

"On a boundary point the derivative must be zero because it's still an optimum." — why wrong?

The 1-D proof needs to step in both

+ h

and

- h

; at a boundary one of those directions leaves the domain, so the inequality only holds on one side and the derivative need not vanish (Figure 4).

f (x, y) = x^{2} - y^{2}

has

\nabla f = 0

at the origin and

H

has a positive and a negative eigenvalue, so it's a minimum in

x

and a maximum in

y

— call it a mixed minimum." — correct the terminology.

Mixed-sign eigenvalues make the Hessian indefinite (bottom-left cell of Figure 3), which is the definition of a saddle point; it is neither a min nor a max, not a "mixed minimum".

"Since

g (t) = f (x^{*} + t e_{i})

has

g^{'} (0) = 0

for every axis vector

e_{i}

, the point is an optimum." — spot the overreach.

Vanishing slopes along the axes

e_{i}

only give

\nabla f = 0

(stationarity), not optimality; a saddle also has zero slope along both axes yet is no optimum (Figure 2, right).

f

is not differentiable at

x^{*}

but has a minimum, so FONC is violated." — is FONC actually broken?

No — FONC is stated for differentiable

f

. If

\nabla f

doesn't exist, FONC simply doesn't apply; the corner (like

∣ x ∣

0

) is a valid non-smooth optimum outside the theorem's scope.

Why questions

Why must the point be interior for FONC to hold?

Because the proof steps in both directions

\pm h

to trap

f^{'}

between

\leq 0

and

\geq 0

(Figure 1); only an interior point has room to move both ways within the domain.

Why does a non-zero gradient rule out an optimum, in one sentence?

Because

D_{u} f = \nabla f \cdot u

is strictly negative for

u = - \nabla f / ∥ \nabla f ∥

, so

f

strictly decreases along

u

— you can always do better, so it's no minimum.

Why do we use the dot product in

D_{u} f = \nabla f \cdot u

rather than some other combination?

The dot product measures how much of the gradient's steepest-increase direction lines up with

u

; it is exactly the first-order change in

f

per unit step along

u

, which is what a directional slope means.

Why does FONC find maxima and saddles as well as minima?

All three are flat to first order — the tangent plane is horizontal (Figure 2) — and the gradient only sees first-order (slope) information, so it cannot distinguish curving-up from curving-down or mixed.

Why is the second-order (Hessian) test needed at all?

The gradient encodes slope but not curvature; deciding min vs max vs saddle requires how the surface bends, which lives in the second derivatives collected in the Hessian (Figure 3).

Why does convexity upgrade "stationary" all the way to "global minimum"?

A convex function lies above each of its tangent planes everywhere; a flat tangent at

x^{*}

then bounds the whole graph from below by

f (x^{*})

, so no point anywhere is lower.

Edge cases

f

is constant everywhere. Where are the stationary points and what are they?

\nabla f = 0

at every point; each is simultaneously a (weak) local min and max, showing FONC can be satisfied by a whole continuum, not a lone point.

The gradient exists but is discontinuous at

x^{*}

, yet equals

0

there. Is

x^{*}

still a valid FONC candidate?

Yes — FONC only requires differentiability at

x^{*}

and

\nabla f (x^{*}) = 0

; continuity of

\nabla f

nearby is not part of the necessary condition.

f (x) = x^{4}

x = 0

: gradient zero, but the Hessian (here

f^{''} (0) = 0

) is also zero. What can you conclude?

The second-order test is inconclusive when the Hessian is only semi-definite/zero (right column of Figure 3); here higher-order analysis shows

0

is a genuine minimum, but the Hessian alone cannot certify it.

A stationary point where the Hessian is positive semidefinite (one eigenvalue exactly

0

). Min guaranteed?

No — positive semidefinite means curvature is upward or flat but never downward; that is consistent with a min or a degenerate saddle (e.g.

f = x^{2} + y^{3}

), so you need higher-order terms to decide (inconclusive column, Figure 3).

Two variables,

\nabla f = 0

, Hessian eigenvalues both negative. What is the point?

A strict local maximum — negative definite means

f

curves downward in every direction (top-left cell of Figure 3), so nearby values are all smaller than

f (x^{*})

A function with no stationary points at all, e.g.

f (x) = x

R

. Does it have an interior optimum?

No — since

\nabla f

is never zero, FONC's necessary condition is never met, so there is no interior max or min (the values run off to

\pm \infty

Recall One-line survival summary

FONC is a filter, not a verdict: $\nabla f = 0$ collects flat points (mins, maxes, saddles, inflections), the Hessian sorts them (Figure 3), boundaries need Lagrange multipliers (Figure 4), and only convexity promotes a stationary point to global champion.

Connections

First-order optimality conditions — gradient = 0 — the parent this bank drills.
Gradient and directional derivatives — the slope tool behind every "why".
Hessian matrix and second-order conditions — resolves the traps FONC leaves open.
Saddle points — the classic false positive.
Convex functions — where stationary really means global.
Lagrange multipliers — the boundary escape hatch.