Optimization — constrained, unconstrained, real-world problems: Visual walkthrough

This is the visual engine behind the whole optimization recipe. The key ideas connect to Critical Points & Fermat's Theorem, Second Derivative & Concavity, and Taylor Series Expansion.

Step 1 — What "slope" even means (the tilt of a hill)

WHAT. Draw a curved path — think of a roller-coaster track drawn as a graph. The horizontal axis is how far along you are (call it $x$ ). The vertical axis is how high you are (call it $f (x)$ , read " $f$ of $x$ " — the height at position $x$ ).

WHY. Before we can say "the slope is zero", we must agree what slope is. Slope = how steeply the track tilts at one spot. A flat road has slope $0$ ; a wall has huge slope.

PICTURE. In the figure, pick a point on the curve and lay a straight ruler so it just kisses the curve there — that ruler is the tangent line. Its tilt is the slope at that point. We measure tilt as rise over run:

$slope = \frac{rise}{run} = \frac{change in height}{change in position} .$

$rise$ — how much you go up (red vertical segment).
$run$ — how much you go across (blue horizontal segment).
their ratio — the steepness. Big up per tiny across = steep.

Figure — Optimization — constrained, unconstrained, real-world problems

Step 2 — Why an interior peak forces the slope to vanish

WHAT. Walk left-to-right over a hilltop that sits strictly inside the interval (not at an edge) and watch the tangent ruler.

WHY. Just before the top you are still climbing, so the tangent tilts up: $f^{'} > 0$ . Just after the top you are descending, so it tilts down: $f^{'} < 0$ . Because we assumed $f^{'}$ is continuous (see the standing assumption), it has the intermediate-value property: it cannot jump straight from a positive value to a negative one — it must pass through $0$ somewhere between. That crossing point is the peak.

PICTURE. Three rulers in the figure: green (up, left of peak), yellow (perfectly flat, at peak), red (down, right of peak).

$climbing f^{'} > 0 ⟶ summit f^{'} = 0 ⟶ descending f^{'} < 0$

left term — positive slope, going up.
middle term — the exact instant of zero tilt, the flat ruler.
right term — negative slope, going down.

Step 3 — The trap: flat does NOT always mean top or bottom

WHAT. Look at $f (x) = x^{3}$ at $x = 0$ .

WHY. We must not blindly say "flat = extremum". The slope of $x^{3}$ is $f^{'} (x) = 3 x^{2}$ , which is $0$ at $x = 0$ — genuinely flat. But $f^{'} = 3 x^{2} \geq 0$ on both sides, so the curve keeps rising the whole time; it merely pauses.

PICTURE. The tangent at the origin is flat (yellow), yet green arrows on both sides point the same way (up-left is below, up-right is above). No summit, no dip.

Here $x^{3}$ happens to be a genuine inflection point — but not merely because $f^{''} (0) = 0$ . What actually makes it an inflection is that the concavity changes sign: $f^{''} (x) = 6 x$ is negative for $x < 0$ (cap) and positive for $x > 0$ (cup). A sign-change of $f^{''}$ on the two sides is the real definition; $f^{''} (c) = 0$ alone is only a hint. So $f^{'} (c) = 0$ makes $c$ only a candidate; we still must classify it.

Step 4 — What curvature looks like (the missing information)

WHAT. A flat tangent tells you the curve pauses; it does not tell you which way the curve bends. We need a second, independent number: how the slope itself is changing.

WHY. Watch the tilt as you move right. Near a valley the tilt goes from negative to positive — the slope is increasing. Near a peak the tilt goes from positive to negative — the slope is decreasing. "Rate at which the slope changes" is exactly the derivative of the derivative.

PICTURE. Two bowls in the figure. The green cup (holds water) has a slope that rises left-to-right. The red cap (spills water) has a slope that falls. This bending is called concavity.

Step 5 — Building the classification test with Taylor's ruler

WHAT. We zoom into a critical point $c$ and approximate the height nearby with a simple polynomial. This is the Taylor Series Expansion — it replaces a messy curve with the best-fitting parabola near one point.

WHY this tool and not another? We want the sign of the change $f (c + h) - f (c)$ for a tiny step $h$ . A tangent line (first order) is useless here — at a critical point the line is flat, telling us nothing. The parabola (second order) is the simplest shape that carries curvature information, and curvature is exactly what distinguishes cup from cap. So we keep terms up to $h^{2}$ :

$f (c + h) = f (c) + f^{'} (c) h + \frac{1}{2} f^{''} (c) h^{2} + \dots$

$f (c + h)$ — height a small step $h$ away from $c$ .
$f (c)$ — height right at $c$ (our reference level).
$f^{'} (c) h$ — the tangent-line correction. At a critical point this is $0$ because $f^{'} (c) = 0$ .
$\frac{1}{2} f^{''} (c) h^{2}$ — the parabola correction, the first surviving term.
$\dots$ — even smaller terms, negligible for tiny $h$ .

PICTURE. The figure overlays the true curve (blue) and its fitted parabola (yellow) hugging it at $c$ ; near $c$ they are indistinguishable, so the parabola's shape is the curve's shape.

Step 6 — Reading the sign: the test falls out

WHAT. Set $f^{'} (c) = 0$ in Step 5's formula and look at what remains:

$f (c + h) - f (c) \approx \frac{1}{2} f^{''} (c) h^{2} .$

WHY. Here is the beautiful part. Whether you step left ( $h < 0$ ) or right ( $h > 0$ ), squaring makes $h^{2} > 0$ . And $\frac{1}{2} > 0$ always. So the entire sign of the height change is decided by $f^{''} (c)$ alone — one number rules both sides.

$h^{2}$ — always positive for any nonzero step, left or right.
$\frac{1}{2}$ — positive constant, changes nothing.
$f^{''} (c)$ — the only factor that can be $+$ or $-$ ; it is the verdict.

PICTURE. If $f^{''} (c) > 0$ : both sides sit above $c$ → valley (local min). If $f^{''} (c) < 0$ : both sides sit below $c$ → peak (local max). The figure shows the parabola opening up (green, min) and down (red, max), with the two step arrows $\pm h$ landing higher / lower.

Step 7 — The degenerate cases: $f^{''} (c) = 0$ , and whole flat plateaus

WHAT. Two degeneracies escape Steps 5–6. (a) Both $f^{'} (c) = 0$ and $f^{''} (c) = 0$ : the ruling term is zero and the parabola is flat, so it gives no verdict. (b) A plateau: $f^{'} \equiv 0$ on a whole interval, so the critical points are not isolated — every point of the flat stretch is critical.

WHY. In case (a) we fell back to a flat parabola; in case (b) there is no single "point" to test. In both, the reliable move is to watch the sign of $f^{'}$ on each side directly.

PICTURE — the first-derivative sign test. March a test point across $c$ and record the sign of $f^{'}$ :

$f^{'}$ goes $+ \to -$ : slope climbs then falls → peak (local max).
$f^{'}$ goes $- \to +$ : slope falls then climbs → valley (local min).
$f^{'}$ keeps the same sign (e.g. $+ \to +$ for $x^{3}$ ): no extremum — a flat inflection (concavity flips, but height keeps going one way).

The two named degeneracies, resolved:

$f (x) = x^{3}$ : $f^{'} = 3 x^{2} \geq 0$ on both sides → same sign → inflection, not an extremum. (Confirmed by concavity flipping: $f^{''} = 6 x$ changes sign.)
$f (x) = x^{4}$ : $f^{'} = 4 x^{3}$ is $-$ for $x < 0$ and $+$ for $x > 0$ → sign goes $- \to +$ → genuine minimum, even though $f^{''} (0) = 0$ and concavity does not flip. This is the figure's fourth panel.

Plateaus. If $f^{'} \equiv 0$ across a whole interval, $f$ is constant there. Every plateau point is simultaneously a (weak) local max and min. To decide if the plateau is a summit or a shelf, apply the sign test at its two ends: check the sign of $f^{'}$ just to the left of the left edge and just to the right of the right edge — same logic, applied to the block rather than a point.

The one-picture summary

Every idea on one graph. A single wavy curve carries: a valley (green, $f^{'} = 0, f^{''} > 0$ ), a peak (red, $f^{'} = 0, f^{''} < 0$ ), and a flat inflection (yellow, $f^{'} = 0$ , concavity flips). Under each, the tiny tangent ruler is drawn flat, and the local best-fit parabola shows the verdict. The margin note reminds you that endpoints and plateaus need the separate sign/boundary check. This is the entire decision procedure of the parent recipe in one frame: find where the ruler goes flat, then read which way the bowl faces — and never forget the edges.

Recall Feynman retelling — the whole walkthrough in plain words

Imagine a roller-coaster. The slope is just how tilted the track feels under your feet — uphill is positive, downhill is negative, flat is zero. To reach the very top of a hill in the middle of the track you have to stop going up before you start going down, and since the tilt changes smoothly (never teleports), for one instant the track is dead flat: that's "the derivative is zero." But watch out — if the top is at the very end of the track, you can still be tilting uphill when you run out of track, so ends need checking on their own. And flatness alone is a fake-out: a track can go flat and keep climbing (that's $x^{3}$ , a shelf, not a summit). So you need a second clue: which way the track curves — cup means you're in a dip, dome means a top. The math trick is to pretend the track near your spot is a simple parabola (Taylor's idea): once the flat part cancels, only $\frac{1}{2} f^{''} h^{2}$ is left, and $h^{2}$ is always positive, so the sign of the curvature alone tells cup or dome. If even the curvature is zero, stop guessing and just watch whether the slope flips sign as you walk across: plus-then-plus is an inflection ( $x^{3}$ ), minus-then-plus is a valley ( $x^{4}$ ). And if the track is flat over a whole stretch — a plateau — just check the tilt at its two ends.

Recall Quick self-check

Why must a smooth interior extremum have $f^{'} (c) = 0$ ? ::: Because $f^{'}$ is continuous, it slides from one sign to the other and must pass through zero (intermediate-value property). Where can an extremum have $f^{'} \neq = 0$ ? ::: At an endpoint / boundary of the interval — the zero-slope test only governs interior points. What single number decides cup vs cap once $f^{'} (c) = 0$ ? ::: The sign of $f^{''} (c)$ , because $\frac{1}{2} h^{2} > 0$ always. Does $f^{''} (c) = 0$ by itself mean an inflection? ::: No — an inflection requires $f^{''}$ to change sign across $c$ ; $x^{4}$ has $f^{''} (0) = 0$ but is a genuine minimum. What do you do when both $f^{'} (c) = 0$ and $f^{''} (c) = 0$ ? ::: Use the first-derivative sign test on the two sides. How do you classify a whole plateau ( $f^{'} \equiv 0$ on an interval)? ::: Check the sign of $f^{'}$ just outside its two ends. What is $x^{3}$ at $0$ ? ::: A flat inflection — critical, concavity flips, but not a max or min.