Foundations — Lagrange multipliers — one and two constraints

This page assumes nothing. Before you can read $\nabla f=\lambda \nabla g$, you must know what each squiggle is and what it looks like. We build them one at a time, each on top of the last.

1. Points, functions, and "a landscape"

Picture: a dot on graph paper (2D) or floating in a room (3D). Nothing more.

Picture: think of $f(x,y)$ as the altitude of a hill above the floor position $(x,y)$. Every floor spot gets one height. That is the "landscape" we keep referring to.

Why the topic needs it: the whole game is "make $f$ as big or small as possible." Without a value-assigning function there is nothing to optimize.

2. Level sets — reading a hill without 3D

Standing inside a 3D hill is hard to draw. So we flatten it.

Picture: rings around a hilltop. Each ring is "all the places at exactly 500 m," the next ring "exactly 600 m," and so on. Crowded rings = steep; spread-out rings = gentle.

Why the topic needs it: the parent page's climax is that at the optimum the constraint path is tangent to a level curve of $f$. You cannot see that without knowing what a level curve is. See Level sets and contours.

3. The gradient $\nabla f$ — the arrow of steepest climb

Here is the star symbol.

Why this tool and not a plain derivative? Plain derivatives are for one input. Our landscape has several inputs $(x,y,z,\dots )$; the partial derivative lets us ask "rate of change in this one direction" separately, which we then bundle together.

Two facts you must feel, not just read:

1. $\nabla f$ points in the direction of steepest ascent — straight uphill. Its length says how steep.
2. $\nabla f$ is perpendicular to the level curve through that point. (Uphill is 90° across the rings, never along them.)

Why the topic needs it: $\nabla f$ is literally "which way does $f$ increase fastest and how fast." Optimization is about killing that increase in the allowed directions. Full detail lives in Gradient and directional derivative.

4. Constraints $g=0$ — the fence you must stay on

Picture: a fence drawn on the floor of the hill. You may walk only on the fence. $g(x,y)=x^2+y^2-1$ with $g=0$ means "stay on the circle of radius 1."

Why the topic needs it: "constrained optimization" = best height on the fence, not the best height anywhere. The fence is the whole reason ordinary calculus (find where $\nabla f=0$) is not enough.

5. Tangent and normal — "along the path" vs "across the path"

To say "$f$ stops rising along the path," we need two directions cleanly separated.

Picture: stand on the fence. "Forward along the fence" = tangent $\mathbf{T}$. "Step off sideways over the fence" = normal $\nabla g$. Details: Tangent planes and normal vectors.

Why the topic needs it: the entire derivation is "at the best point, $\nabla f$ has no tangent part — it lies fully along the normal $\nabla g$." That sentence is meaningless until tangent and normal are pinned down.

6. The dot product — measuring "along" precisely

Why this tool and not another? We keep asking "does $\nabla f$ have any component along the path direction ${\mathbf{r}}^{\prime }$?" The dot product $\nabla f\cdot {\mathbf{r}}^{\prime }$ is precisely "how much of $\nabla f$ points along ${\mathbf{r}}^{\prime }$." Setting it to zero says "none." That's why the parent's Step 2 is $\nabla f\cdot {\mathbf{r}}^{\prime }=0$.

7. The chain rule — how height changes as you walk

Why the topic needs it: this is the bridge. "$f$ stops changing along the path" is $\frac{d}{dt}f(\mathbf{r}(t))=0$; the chain rule turns that into $\nabla f\cdot {\mathbf{r}}^{\prime }=0$, which the dot product reads as "$\nabla f\perp$ every tangent." That is how we conclude $\nabla f$ must be normal.

8. The multiplier $\lambda$ — a single knob for "how parallel"

Picture: $\nabla f$ and $\nabla g$ are two arrows planted at the optimum, lying on the same line. $\lambda$ is "how many $\nabla g$'s make one $\nabla f$" — the ratio of their lengths (with a sign if they point opposite ways).

• $\lambda >0$: $\nabla f$ and $\nabla g$ point the same way.
• $\lambda <0$: they point opposite ways (still the same line).
• $\lambda =0$: $\nabla f=0$ — the constraint isn't doing any work; it's an unconstrained critical point. See Unconstrained optimization — critical points.

Why the topic needs it: it converts the geometric statement "parallel" into a solvable equation. And, as Dual problem and shadow prices shows, $\lambda =d{f}^{\ast }/dc$ measures how much the best value improves when you nudge the fence.

9. Two constraints — why a second multiplier $\mu$ appears

With two fences $g=0$ and $h=0$, you can only stand where they cross: the intersection curve. Along that single curve there is one tangent $\mathbf{T}$, but two normals $\nabla g$ and $\nabla h$ span the whole "sideways" plane. For $\nabla f$ to have no along-curve part, it must lie in that plane — a blend of both normals: $\nabla f=\lambda \nabla g+\mu \nabla h.$ The second scalar $\mu$ is the second multiplier — one knob per fence. This connects forward to inequality versions in KKT conditions.

Prerequisite map

Equipment checklist

Test yourself — you should be able to answer each before reading the parent derivation.