Visual walkthrough — First-order optimality conditions — gradient = 0

---

Step 1 — What does "slope" even mean?

WHAT. Before any calculus, picture a smooth hill drawn as a curve. Pick a point on it. Zoom in. As you zoom, the curve looks more and more like a straight line — its tangent line. The slope of that line is a single number: how many steps up you go for one step to the right.

WHY. Optimisation is the question "can I get lower?" — and "can I get lower" is answered entirely by the slope under your feet. If the ground tilts, one direction goes down. So slope is the whole game; we must pin it down first.

PICTURE. In the figure below, the black curve is the hill. The magenta line is the tangent at the marked point. Its slope is the ratio (rise)/(run) shown by the orange triangle.

---

Step 2 — Turning slope into a number we can compute

WHAT. To measure the slope at a point $x^{\ast }$, take a tiny horizontal step of size $h$ and see how much height changes. Height at the new point is $f(x^{\ast }+h)$; height at the old point is $f(x^{\ast })$. Their difference over the step is the difference quotient:

$$\underset{\text{rise over run}}{\underbrace{\frac{\stackrel{\text{height after step}}{\overbrace{f(x^{\ast }+h)}}-\stackrel{\text{height now}}{\overbrace{f(x^{\ast })}}}{\underset{\text{how far we stepped}}{\underbrace{h}}}}}$$

WHY. We want the exact slope, but "exact" needs the step to shrink to nothing. The difference quotient is the honest rise-over-run for a step of real width $h$; shrinking $h\to 0$ hands us the tangent's slope — that limit is the derivative $f^{\prime }(x^{\ast })$, which exists precisely because we assumed $f$ differentiable at $x^{\ast }$. We use a limit (and not just plugging $h=0$) because plugging $h=0$ gives $0/0$, which is meaningless — the limit is the tool built exactly to rescue that.

PICTURE. Two dots on the curve joined by a violet secant line; as the right dot slides toward the left dot the secant tips over into the magenta tangent.

---

Step 3 — The setup: what "local minimum" promises us

WHAT. Say $x^{\ast }$ is a local minimum: no nearby point is lower. Formally, there is a small radius $\delta >0$ so that for every step $h$ with $|h|<\delta$,

$$\underset{\text{any nearby height}}{\underbrace{f(x^{\ast }+h)}}\ge \underset{\text{our valley floor}}{\underbrace{f(x^{\ast })}}.$$

The bar $|h|<\delta$ means "$h$ is smaller than $\delta$ in size, whether $h$ is positive or negative" — steps to the right and to the left are both allowed.

WHY. This single inequality is the only fact we have about a minimum. Everything below is squeezed out of it. Notice the crucial word interior: because $x^{\ast }$ sits inside the domain, both $+h$ and $-h$ stay legal. Hold that thought — Step 6 is where it bites.

PICTURE. A valley with a flat floor; a shaded band of width $2\delta$ around $x^{\ast }$; every point of the curve in that band sits at or above the dashed floor line.

---

Step 4 — Poke to the RIGHT (a one-sided limit, $h\to 0^{+}$)

WHAT — first, what does $h\to 0^{+}$ mean? A one-sided limit from the right, written ${\lim }_{h\to 0^{+}}$, asks: as $h$ shrinks toward $0$ but stays strictly positive (approaching only from the right, $h=0.1,0.01,0.001,\dots$), what value does the quantity home in on? It is the ordinary idea of a limit, but restricted to one side of $0$. We split into sides precisely because "step right" ($h>0$) and "step left" ($h<0$) will tell us different things.

Now take steps to the right, so $h>0$, and read the difference quotient piece by piece:

$$\frac{\stackrel{\ge 0\text{(min: nearby}\ge \text{floor)}}{\overbrace{f(x^{\ast }+h)-f(x^{\ast })}}}{\underset{>0\text{(stepping right)}}{\underbrace{h}}}\ge 0.$$

Top is $\ge 0$ because a minimum means every neighbour is at least as high. Bottom is $>0$ because we stepped right. A non-negative number over a positive number is non-negative. Letting $h\to 0^{+}$:

$$f^{\prime }(x^{\ast })=\underset{h\to 0^{+}}{\lim }(\cdots )\ge 0.$$

(Differentiability lets us call this one-sided limit $f^{\prime }(x^{\ast })$ — both one-sided limits equal the single derivative.)

WHY. A limit of numbers that are all $\ge 0$ cannot suddenly turn negative — it lands $\ge 0$. So from the right side alone, the slope is non-negative: the tangent can't dip down as we look rightward.

PICTURE. The right-hand track of dots creeping toward $x^{\ast }$ from the right; the secant slopes are all $\ge 0$, shown in orange.

---

Step 5 — Poke to the LEFT (the mirror one-sided limit, $h\to 0^{-}$)

WHAT. The one-sided limit from the left, ${\lim }_{h\to 0^{-}}$, is the mirror of Step 4: $h$ shrinks toward $0$ but stays strictly negative ($h=-0.1,-0.01,-0.001,\dots$), approaching only from the left. Step left, so $h<0$:

$$\frac{\stackrel{\ge 0\text{(still}\ge \text{floor)}}{\overbrace{f(x^{\ast }+h)-f(x^{\ast })}}}{\underset{<0\text{(stepping left)}}{\underbrace{h}}}\le 0.$$

Top is still $\ge 0$ (a minimum is a minimum in both directions). But now the bottom is negative. A non-negative number divided by a negative number is $\le 0$. So

$$f^{\prime }(x^{\ast })=\underset{h\to 0^{-}}{\lim }(\cdots )\le 0.$$

WHY. Only the sign of $h$ flipped, which flips the whole quotient's sign. From the left, the same derivative is forced to be non-positive.

PICTURE. Left-side secants, mirror image of Step 4, all tilting downward or flat in violet as the left dot slides toward $x^{\ast }$.

---

Step 6 — The squeeze: two inequalities crush into equality

WHAT. Because $f$ is differentiable at $x^{\ast }$, the left-side and right-side one-sided limits are the same number $f^{\prime }(x^{\ast })$. Step 4 said it is $\ge 0$; Step 5 said it is $\le 0$. A number that is both cannot budge:

$$f^{\prime }(x^{\ast })\ge 0\text{and}{f}^{\prime }(x^{\ast })\le 0⟹\boxed{f^{\prime }(x^{\ast })=0.}$$

WHY. This is why we needed both directions, and why the point must be interior. On a boundary you can only poke one way — you'd get just Step 4 or just Step 5, never both, and the slope could stay nonzero (pointing outward). The two-sided poke is the entire engine.

PICTURE. A number line for the value $f^{\prime }(x^{\ast })$: an orange arrow from the right pins it $\ge 0$, a violet arrow from the left pins it $\le 0$; they collide at a single magenta dot on $0$.

---

Step 7 — Climbing to many variables, one axis at a time

WHAT. Now $f$ eats several inputs, e.g. $f(x,y)$ — a landscape, not a curve. Freeze every coordinate except one. Sliding only along the $x_i$-axis carves out a 1-D slice — call its height $g(t)=f(x^{\ast }+t{e}_i)$, where $e_i$ is the unit step along axis $i$ and $t$ is how far you've slid.

If $x^{\ast }$ is the lowest point of the whole landscape, it is certainly the lowest point along that single slice, so $t=0$ minimises $g$. By Step 6, $g^{\prime }(0)=0$. And $g^{\prime }(0)$ is exactly the partial derivative $\partial f/\partial x_i$ — the slope of $f$ if you only wiggle the $i$-th input.

WHY. A many-variable minimum is a minimum along every one-dimensional line through it, so each slice hands us one flat-slope equation. Do this for every axis and all partials vanish. We now bundle those partial slopes into a single arrow, the gradient $\nabla f$ — the vector whose $i$-th entry is $\partial f/\partial x_i$ (see Gradient and directional derivatives). Then "all partials zero" is exactly "the gradient arrow has zero length":

$$\underset{\text{slope along axis }1}{\underbrace{\frac{\partial f}{\partial x_1}(x^{\ast })}}=\cdots =\underset{\text{slope along axis }n}{\underbrace{\frac{\partial f}{\partial x_n}(x^{\ast })}}=0⟺\nabla f(x^{\ast })=0.$$

PICTURE. A bowl-shaped surface; two perpendicular slice-curves cut through the bottom, each showing a flat tangent at the base — one along $x$, one along $y$.

---

Step 8 — The deepest reason: no downhill direction survives

WHAT — first, two tools we must define.

Tool A — the length (norm) of a vector. The gradient $\nabla f(x^{\ast })$ (built in Step 7) is an arrow with a direction and a length. Its length is written with double bars, $\Vert \cdot \Vert$, and computed by Pythagoras on its components:

$$\Vert v\Vert =\sqrt{v_1^2+v_2^2+\cdots +v_n^2}\ge 0.$$

Two facts we use: the norm is never negative, and it is zero only for the zero vector (a zero-length arrow is the arrow that points nowhere). So "$\Vert \nabla f\Vert >0$" is just shorthand for "$\nabla f$ is a genuine, nonzero arrow."

Tool B — the directional derivative. The gradient points in the direction of steepest increase. The directional derivative $D_{u}f(x^{\ast })$ answers: if I walk in a chosen unit direction $u$ (an arrow of length $1$), how fast does $f$ change? The formula is a dot product:

$$D_{u}f(x^{\ast })=\underset{\text{steepest-up arrow}}{\underbrace{\nabla f(x^{\ast })}}\cdot \underset{\text{unit walking direction}}{\underbrace{u}}.$$

The dot product $\cdot$ multiplies the two vectors' lengths by the cosine of the angle between them: it is largest when $u$ points with the gradient, zero when $u$ is perpendicular, and most negative when $u$ points against the gradient.

Now the argument. Suppose, for contradiction, $\nabla f(x^{\ast })\ne 0$, i.e. $\Vert \nabla f(x^{\ast })\Vert >0$. Deliberately walk straight downhill, choosing the unit direction

$$u=-\frac{\nabla f(x^{\ast })}{\Vert \nabla f(x^{\ast })\Vert }(\text{gradient reversed, shrunk to length }1).$$

Plug into the formula — dotting a vector with a scaled copy of itself gives its length squared over its length:

$$D_{u}f(x^{\ast })=\nabla f\cdot \left(-\frac{\nabla f}{\Vert \nabla f\Vert }\right)=-\frac{\Vert \nabla f{\Vert }^2}{\Vert \nabla f\Vert }=-\Vert \nabla f(x^{\ast })\Vert <0.$$

A negative slope means $f$ decreases as you walk — so $x^{\ast }$ was not a minimum. Contradiction. The only escape: $\nabla f(x^{\ast })=0$.

WHY. This says why zero gradient is unavoidable: a nonzero gradient is literally an arrow labelled "improvement lives this way." A genuine optimum can't leave such an arrow lying around.

PICTURE. At a non-optimal point, the magenta gradient arrow points uphill; the orange $u=-\nabla f$ arrow points downhill along the surface, tracing a path that visibly drops — proving the point was beatable.

---

Step 9 — The traps: flat is not the same as bottom

WHAT. Flatness ($\nabla f=0$) is necessary, never sufficient. Three different shapes all pass the test:

• Minimum: flat, and rising in every direction (a bowl).
• Maximum: flat, and falling in every direction (a dome). Same proof — just swap $\ge$ for $\le$.
• Saddle: flat, yet rising along one axis and falling along another. Example $f(x,y)=x^2-y^2$: along $x$ it climbs, along $y$ it drops, but $\nabla f(0,0)=0$ all the same. See Saddle points.

Even in 1-D, $f(x)=x^3$ has $f^{\prime }(0)=0$ yet $0$ is an inflection, neither peak nor valley.

WHY. Our proof only ruled out "a direction that improves things immediately." A saddle has directions that improve — just not right at the flat point along the tested axis. That is exactly why the Hessian second-order test exists: to sort these three apart.

PICTURE. Three mini-landscapes side by side — bowl, dome, saddle — each with a flat magenta tangent plane at the centre dot, showing identical flatness but wildly different shapes.

---

The one-picture summary

Everything at once: the two-sided poke traps the slope at zero (1-D), stacking that over all axes gives $\nabla f=0$ (many-D), and the flat point is only a finalist — bowl, dome, or saddle still to be decided.

Recall Feynman: the whole walkthrough in plain words

You're standing somewhere on a smooth landscape and you want to know if you're at the lowest spot. You do one simple experiment: nudge your foot right — if you're at a min, the ground can't drop, so the right-slope is at least zero ($\ge 0$). Nudge left — by the very same reasoning the ground can't drop that way either, but "not dropping to the left" means the slope read leftward is at most zero ($\le 0$). Here's the punchline: the ground has one slope (that's what "smooth / differentiable" guarantees), and the only number that is simultaneously "not below zero" and "not above zero" is exactly zero. So the ground is flat. In a full landscape you just repeat this nudge along every compass axis, and every one comes back flat — that bundle of flat slopes is what $\nabla f=0$ packs into a single arrow of zero length. The deepest way to see it: if the ground weren't flat, the gradient would be a real arrow shouting "downhill is that way," and you'd follow it to somewhere lower — so you couldn't have been at the bottom. The catch, drawn in the last figure: flat ground shows up at the top of a hill and in the middle of a horse-saddle too. Flat is a clue, not a verdict. To read the verdict you bend down and check the curvature — and that is the Hessian's job.

---

Connections

• Parent: FONC (Hinglish) — the result this page derives visually.
• Gradient and directional derivatives — Steps 7–8 run on it.
• Hessian matrix and second-order conditions — settles the Step 9 traps.
• Saddle points — the middle trap made explicit.
• Lagrange multipliers — what to do when Step 6's "interior" fails on a boundary.
• Gradient descent — literally walks the Step 8 downhill arrow.
• Convex functions — the one world where flat does guarantee the global bottom.

Concept Map