Critical points and saddle points: Visual walkthrough

We will use a running one-dimensional idea first (because you can draw it), then climb to two dimensions where the saddle actually appears.

Step 1 — What does "flat ground" even mean?

WHAT. Picture a smooth hill as a curve $y = f (x)$ : the horizontal axis $x$ is where you stand, the vertical axis $f (x)$ is how high the ground is. The slope at a point is the steepness of the tiny straight line that just kisses the curve there (the tangent).

WHY. Before we can talk about minima or saddles, we need the one object that measures "am I on a slope or flat?": the slope itself. We write the slope of $f$ at $x$ as $f^{'} (x)$ — read " $f$ -prime of $x$ ". It answers the question "if I take a tiny step to the right, how much does my height change per unit step?"

PICTURE. In the figure, the black curve is the ground. At the orange point the tangent line tilts — you are on a slope. At the green point the tangent is perfectly horizontal — that is a critical point.

$f^{'} (x) = how fast height changes per step slope of the tangent line at x$

At the green point: $f^{'} (x^{*}) = 0 ⟺ tangent is horizontal$

Here $x^{*}$ (read "x-star") is just our name for the special flat spot.

Step 2 — Flat is not enough: three flat spots that mean three different things

WHAT. Three different curves each have $f^{'} (x^{*}) = 0$ at the same spot, yet one is a bottom (minimum), one is a top (maximum), and one just flattens for an instant then keeps going (inflection).

WHY. This is the whole reason we need something beyond the slope. The slope being zero cannot distinguish these three — they all have a horizontal tangent. We must measure how the slope is changing, i.e. whether the curve bends up or bends down.

PICTURE. Blue bowl bends upward → the slope goes from negative to positive → a minimum. Red dome bends downward → slope goes from positive to negative → a maximum. Gray curve flattens but keeps rising → an inflection, no extremum at all.

Recall Why can't the slope alone tell them apart?

Because at $x^{*}$ all three have slope $::: 0$ ; the difference lives in how the slope changes as you pass through — that is the second derivative.

Step 3 — The second derivative: measuring bend

WHAT. The second derivative $f^{''} (x)$ (read " $f$ -double-prime") is the slope of the slope: how fast the tangent's tilt is changing as you walk.

WHY. We need a number that says "up-bowl" or "down-bowl". Watch the tangent line as you cross $x^{*}$ :

If the tangent rotates counter-clockwise (slope increasing), the curve is a bowl → $f^{''} > 0$ .
If it rotates clockwise (slope decreasing), the curve is a dome → $f^{''} < 0$ .

This is exactly the question we asked in Step 2, now answered by a single sign.

PICTURE. The figure shows three tiny tangent arrows at $x^{*} - h$ , $x^{*}$ , $x^{*} + h$ . When they fan open upward the bend is positive; downward, negative.

$f^{''} (x^{*}) = how the tangent rotates \frac{change in slope}{step} ⎩ ⎨ ⎧ > 0 < 0 = 0 bowl (minimum) dome (maximum) inconclusive — go higher$

Step 4 — Where these numbers come from: the Taylor picture

WHAT. Zoom into any smooth curve near $x^{*}$ and it looks like a parabola. The recipe that says so is the Taylor expansion: $f (x^{*} + h) \approx height now f (x^{*}) + slope \times step f^{'} (x^{*}) h + bend \times step^{2} \frac{1}{2} f^{''} (x^{*}) h^{2}$

WHY. This equation is the engine behind Steps 1–3. It splits the height change into a flat part, a tilt part (first derivative), and a bend part (second derivative). At a critical point the tilt part dies ( $f^{'} (x^{*}) = 0$ ), so the leftover bend part alone decides up or down: $f (x^{*} + h) - f (x^{*}) \approx \frac{1}{2} f^{''} (x^{*}) h^{2}$ Since $h^{2} \geq 0$ always, the sign of $f^{''} (x^{*})$ is the sign of the height change — bowl or dome. This is why the second derivative is the right tool and no lower-order one will do.

PICTURE. The blue curve is the true $f$ ; the orange dashed parabola is the Taylor approximation. Near $x^{*}$ they hug; far away they drift. See Taylor Series for the full construction.

Step 5 — Climbing to 2-D: the flat spot needs every direction

WHAT. Now stand on a real terrain $z = f (x, y)$ : two ground coordinates $x, y$ , height $z$ . Being flat now means flat along both directions at once. The slopes in each direction are the partial derivatives $\partial f / \partial x$ and $\partial f / \partial y$ ; stacking them gives the gradient: $\nabla f = (tilt east \frac{\partial f}{\partial x}, tilt north \frac{\partial f}{\partial y})$

WHY. A critical point in 2-D means $\nabla f = (0, 0)$ : no tilt east–west and no tilt north–south. If even one component is nonzero, you're on a slope somewhere and it isn't flat. This is exactly the 1-D flatness of Step 1, now demanded in both compass directions.

PICTURE. The surface $f (x, y) = x^{2} - y^{2}$ : the arrows show tilt is zero at the center along both axes even though the surface is clearly not a bowl. That mismatch is the saddle we are hunting.

Step 6 — The Hessian: the bend-in-every-direction machine

WHAT. In 1-D one number ( $f^{''}$ ) captured bend. In 2-D bend depends on which way you walk, so we need a small table. The Hessian collects all four second derivatives: $H = \frac{\partial ^{2} f}{\partial x ^{2}} \frac{\partial ^{2} f}{\partial y \partial x} \frac{\partial ^{2} f}{\partial x \partial y} \frac{\partial ^{2} f}{\partial y ^{2}}$

Read it term by term: top-left $=$ bend walking east–west; bottom-right $=$ bend walking north–south; the off-diagonals $=$ how east-bend and north-bend twist together.

WHY. The 2-D Taylor expansion needs it. With $d = (d_{x}, d_{y})$ a tiny step and $\nabla f = 0$ at the critical point: $f (x^{*} + d) - f (x^{*}) \approx \frac{1}{2} bend along step d d^{⊤} H d$ This is the direct sibling of the $\frac{1}{2} f^{''} h^{2}$ from Step 4. The quantity $d^{⊤} Hd$ (a quadratic form) is the height change — and its sign, over all directions $d$ , is the verdict.

PICTURE. For $f = x^{2} - y^{2}$ the machine reads: $H = (20 0 - 2), d^{⊤} Hd = 2 d_{x}^{2} - 2 d_{y}^{2}$ Walk east ( $d_{y} = 0$ ): $+ 2 d_{x}^{2} > 0$ , ground rises. Walk north ( $d_{x} = 0$ ): $- 2 d_{y}^{2} < 0$ , ground falls. Up one way, down another → saddle.

Step 7 — Eigenvalues: the pure up/down directions

WHAT. The off-diagonal terms make $d^{⊤} Hd$ hard to read in general. The eigenvalues $λ_{1}, λ_{2}$ of $H$ are the bends along $H$ 's own natural axes (its eigenvectors), where the twist disappears and each direction is pure bowl or pure dome. See Eigenvalues and Eigenvectors.

WHY. Rotate your coordinates onto those special axes and the quadratic form becomes clean: $d^{⊤} Hd = λ_{1} u_{1}^{2} + λ_{2} u_{2}^{2}$ where $u_{1}, u_{2}$ are the step measured along the eigenvector axes. Because $u_{1}^{2}, u_{2}^{2} \geq 0$ , the signs of $λ_{1}, λ_{2}$ alone decide everything:

Signs of eigenvalues	Every direction bends…	Verdict
all $λ_{i} > 0$	up	local minimum (bowl)
all $λ_{i} < 0$	down	local maximum (dome)
mixed signs	up some, down some	saddle point
some $λ_{i} = 0$	flat in a direction	degenerate — go higher

PICTURE. The two colored arrows are the eigenvector axes of $x^{2} - y^{2}$ : blue is the $λ = + 2$ (up) direction, red is the $λ = - 2$ (down) direction. Every escape route from the saddle is a mix of these two.

Step 8 — The degenerate edge case (a zero eigenvalue)

WHAT. What if $λ_{2} = 0$ ? Then along that eigenvector the bend part $λ_{2} u_{2}^{2} = 0$ — the Taylor quadratic says nothing in that direction, just like $f^{''} = 0$ back in Step 3.

WHY. A zero eigenvalue means the surface is momentarily flat along a whole direction; whether it eventually rises or falls there depends on cubic or higher terms. The Hessian test is inconclusive — you must look further (exactly the $x^{3}$ situation from Step 3, now in 2-D).

PICTURE. The "monkey-saddle-ish" trough $f (x, y) = x^{2}$ has $H = (2000)$ : a bowl in $x$ , a dead-flat channel in $y$ . Along that flat channel the second-order test gives no answer.

The one-picture summary

Everything above is one idea repeated: flat gradient locates the spot; the sign-pattern of the Hessian's eigenvalues names it. The final figure stacks the three verdicts (bowl, saddle, dome) with their eigenvalue signs side by side.

Recall Feynman retelling — say it back in plain words

You're a blindfolded hiker. First you feel: is the ground flat here? If any direction tilts, walk downhill — you're not special. When it's flat in every direction, you've hit a critical point, but you still don't know if it's a valley, a peak, or a mountain pass.

So you probe the curvature. In 1-D, one number — the second derivative — tells you bowl (positive) or dome (negative). In many dimensions, curvature depends on which way you step, so you collect all the bends into the Hessian. Its eigenvalues are the bends along its own private axes, where each direction is a clean bowl or clean dome. If all eigenvalues are positive, every direction climbs away — you're in a valley. All negative — you're on a peak. Mixed — some directions rise, some fall — a saddle, with a downhill escape along any negative-eigenvalue direction. If one eigenvalue is exactly zero, curvature is silent there and you must look at higher terms. That's the whole story: gradient finds it, eigenvalues judge it.

See also: Convex Optimization (where every critical point is guaranteed a minimum, so all eigenvalues stay positive everywhere).