Lagrange multipliers — one and two constraints: Visual walkthrough

Step 1 — What "optimize on a constraint" even means

WHAT. We have a landscape: a function $f (x, y)$ that gives a height at every point of the flat $(x, y)$ plane. We are told we may only stand on a particular path — a curve drawn on the ground, described by an equation $g (x, y) = 0$ . We want the highest (or lowest) height reachable while staying on that path.

WHY this framing. Without a rule, the best point is wherever the whole landscape peaks. The rule $g = 0$ forbids most of the ground. So the answer changes: the constrained best is usually not the free best.

PICTURE. Look at the figure. The wavy shading is $f$ : brighter = higher. The yellow curve is the allowed path $g = 0$ . We are only permitted to walk along the yellow curve, and we hunt for its highest-brightness point (the red dot).

Figure — Lagrange multipliers — one and two constraints

Related idea: the shading here is a picture of Level sets and contours flattened into brightness.

Step 2 — Turn the path into a moving dot: the parametrization $r (t)$

WHAT. To talk about walking along the yellow curve, we describe our position by a single dial $t$ (think of it as time or distance walked). At each $t$ we are at a point

$r (t) = (x (t), y (t)) .$

Here $r (t)$ is the position vector (an arrow from the origin to where we stand), $x (t)$ and $y (t)$ are our two coordinates, each changing as we turn the dial $t$ .

WHY. A constraint is a whole curve — infinitely many points. That is hard to optimize over directly. But once we ride along it with one dial $t$ , the height becomes an ordinary one-variable function $f (r (t))$ , and we already know how to find the top of a one-variable hill: its slope is zero.

PICTURE. The red dot slides along the yellow curve as $t$ increases. The little green arrow $r^{'} (t)$ is the direction we are heading — the tangent to the path. It always points along the curve, never off it.

Step 3 — At the best spot, the height stops changing

WHAT. Let $x^{*} = r (t_{0})$ be the highest point we can reach on the curve. Because the restricted height $f (r (t))$ is now just a normal function of one dial $t$ , its top happens where its slope vanishes:

$how fast height changes as we walk \frac{d}{d t} f (r (t))_{t_{0}} = 0.$

WHY. This is the plain one-variable fact: at a maximum the graph is momentarily flat — go a hair forward or back and you are not higher. Nothing multivariable yet; we simply transported that fact onto the curve.

PICTURE. Below the map we plot the "elevation profile" — height versus $t$ as you march along the yellow path. The peak of that profile sits directly above the red dot on the map. At the peak the profile is flat: slope $= 0$ .

Step 4 — Decode the slope with the chain rule → meet the gradient

WHAT. How does $f$ change as we walk? Two things combine: how steep $f$ is in each direction, and which way we are stepping. The chain rule bundles them:

$\frac{d}{d t} f (r (t)) = steepness of the landscape \nabla f \cdot our step direction r^{'} (t) .$

Let us unpack the brand-new symbol $\nabla f$ .

The dot " $\cdot$ " is the dot product: it multiplies two arrows and returns a single number measuring how much they agree in direction. It is exactly the right tool here because the height-change-per-step is precisely "how much of my step goes uphill" — which is agreement between my step $r^{'}$ and the uphill arrow $\nabla f$ .

WHY the dot product and not something else? We need a rule that is big when I walk straight uphill, zero when I walk across the slope, negative when I walk downhill. The dot product does exactly this: $\nabla f \cdot r^{'} = ∣\nabla f ∣ ∣ r^{'} ∣ cos θ$ , where $θ$ is the angle between the uphill arrow and my step. Walk along the slope ( $θ = 9 0^{\circ}$ ) → $cos 9 0^{\circ} = 0$ → no height change.

PICTURE. At the red dot we draw the blue gradient arrow $\nabla f$ (steepest uphill) and the green step arrow $r^{'}$ (along the path). Their overlap — the shadow of one onto the other — is the rate of height change.

Step 5 — The consequence: $\nabla f$ must be perpendicular to the path

WHAT. Combine Step 3 (slope $= 0$ ) with Step 4 (slope $= \nabla f \cdot r^{'}$ ):

$\nabla f (x^{*}) \cdot r^{'} (t_{0}) = 0.$

A dot product of two nonzero arrows is zero exactly when they are perpendicular. So at the best point, the uphill arrow $\nabla f$ is perpendicular (⟂) to our step direction $r^{'}$ .

WHY it must hold for every path. We could have walked along the curve in either direction (forward or backward $t$ ); both are tangent directions. Since $\nabla f \cdot r^{'} = 0$ for the tangent, $\nabla f$ has no component along the curve at all. If it did have a sideways-along-path component, we could step that way and climb higher — contradicting "best point".

PICTURE. At the red dot, the blue $\nabla f$ arrow stands at a clean right angle to the yellow curve. We shade the tiny tangent segment; the blue arrow leans neither forward nor back along it.

Step 6 — Meet the other perpendicular arrow: $\nabla g$

WHAT. The constraint curve is the set $g = 0$ . As we walk it, $g$ never changes (it stays $0$ ). Differentiate that fact along the path:

$\frac{d}{d t} g (r (t)) = \nabla g \cdot r^{'} (t) = 0.$

The left side is $0$ because $g$ is constant ( $= 0$ ) on the whole curve. So $\nabla g \cdot r^{'} = 0$ : the gradient of $g$ is also perpendicular to the path.

WHY. $\nabla g$ points in the direction $g$ increases fastest — straight off the level curve $g = 0$ , toward larger $g$ . Naturally that is at right angles to the curve, which is the direction $g$ stays put. This is exactly the "normal vector to a surface" idea, see Tangent planes and normal vectors.

PICTURE. Same red dot, now with the red arrow $\nabla g$ drawn — pointing directly off the yellow curve, perpendicular to it, alongside the blue $\nabla f$ .

Step 7 — Two perpendiculars to one curve must be parallel: the Lagrange condition

WHAT. In the plane, a curve has only one perpendicular direction (a single line of "off the curve"). Step 5 put $\nabla f$ on that line; Step 6 put $\nabla g$ on that same line. Two arrows on one line are scalar multiples of each other:

$\nabla f = λ \nabla g$

Term by term:

$\nabla f$ ::: uphill arrow of the objective (must vanish along the path).
$\nabla g$ ::: perpendicular-to-constraint arrow.
$λ$ ::: the Lagrange multiplier — the single number saying how many times longer $\nabla g$ must be scaled to match $\nabla f$ . Its sign says whether they point the same or opposite way.

Plus we must not forget the location equation itself:

$g (x) = 0.$

WHY $λ$ can be any sign or zero.

$λ > 0$ : $\nabla f$ and $\nabla g$ point the same way.
$λ < 0$ : they point opposite ways (both still perpendicular to the curve — perpendicular has two directions).
$λ = 0$ : $\nabla f = 0$ , i.e. the free (unconstrained) critical point happens to lie on the curve; the constraint is "not pushing". This is the bridge to Unconstrained optimization — critical points.

PICTURE. The finale of the derivation: at the optimum the blue and red arrows are collinear — same line, possibly different lengths and directions. Off the optimum (faint dot) they are not aligned, and a green tangential component of $\nabla f$ shows you can still climb.

Step 8 — Degenerate cases you must not skip

WHAT & WHY. The clean picture assumed $\nabla g \neq = 0$ and the curve was smooth. When that breaks, the argument breaks — so extrema can hide there.

Singular point, $\nabla g = 0$ . Then $\nabla g$ has no direction, so " $\nabla f$ parallel to $\nabla g$ " is empty — the equation $\nabla f = λ \cdot 0$ forces $\nabla f = 0$ , which need not happen at the true extremum. Example: a curve with a sharp corner (a cusp). The tip can be the max, yet no $λ$ describes it. You must check these points by hand.
Level curves tangent (the "kissing" view). At the optimum the curve $g = 0$ is tangent to a level curve of $f$ — they touch without crossing. If they crossed, $f$ would be higher on one side, so you'd move there. Tangency is the same fact as "gradients parallel", seen through Level sets and contours.

PICTURE. Left: a cusp where $\nabla g = 0$ and Lagrange is blind. Right: the level curves of $f$ (faint rings) kissing the constraint at the red dot — the geometric twin of $\nabla f ∥ \nabla g$ .

The one-picture summary

Everything at once: the shaded landscape $f$ , the yellow constraint $g = 0$ , the level rings of $f$ kissing it at the red optimum, and the two collinear arrows $\nabla f = λ \nabla g$ standing perpendicular to the curve.

Recall Feynman retelling of the walkthrough

You're hiking on a fenced trail across a hilly field. You want the highest spot you can reach without leaving the trail. So you just walk. As long as the ground still tilts forward along the trail, you keep climbing. You stop where the only remaining tilt is sideways across the trail — walking on gets you nothing. "Uphill points sideways across the trail" means the steepest-uphill arrow ( $\nabla f$ ) is perpendicular to the trail. But the fence's own "straight-off-me" arrow ( $\nabla g$ ) is also perpendicular to the trail. In flat 2-D there's only one perpendicular line, so the two arrows must lie on it together — same line, maybe different lengths: that scaling number is $λ$ . Watch out for kinks in the fence (a sharp corner has no well-defined "off" direction) — there you must look for yourself, because the arrow trick goes silent.

Connections

Lagrange multipliers — one and two constraints — the parent result this page derives visually.
Gradient and directional derivative — $\nabla f$ as steepest ascent, Step 4.
Tangent planes and normal vectors — $\nabla g$ as the surface normal, Step 6.
Level sets and contours — the "kissing level curves" view, Step 8.
Unconstrained optimization — critical points — the $λ = 0$ limiting case, Step 7.
Dual problem and shadow prices — later meaning of $λ$ .