Exercises — Lagrange multipliers — one and two constraints

WHAT we do: just translate words into the standard template. (a) Move everything to one side: $g(x,y)=x^2+y^2-25$, and the constraint is $g=0$. (b) $\nabla f=(3,4)$ (constant — the slopes of a plane never change) and $\nabla g=(2x,2y)$. (c) Lagrange's condition: $(3,4)=\lambda (2x,2y),x^2+y^2=25.$ That is 3 equations ($3=2\lambda x$, $4=2\lambda y$, and the constraint) in 3 unknowns $(x,y,\lambda )$. Recognizing this count is the whole point of L1.

(a) False. On a constraint, $\nabla f$ is usually nonzero; it must point off the surface, parallel to $\nabla g$. Setting $\nabla f=0$ is the unconstrained rule (Unconstrained optimization — critical points). (b) False. $\nabla g$ points perpendicular (normal) to the surface, not along it. (c) True. $\lambda =d{f}^{\ast }/dc$ is the shadow price (Dual problem and shadow prices).

From $3=2\lambda x$ and $4=2\lambda y$ we solve for the coordinates in terms of $\lambda$: $x=\frac{3}{2\lambda },y=\frac{4}{2\lambda }=\frac{2}{\lambda }.$ WHY divide: the two component equations already give $x,y$; we just need one more relation ($\lambda$) which the constraint supplies. Substitute into $x^2+y^2=25$: $\frac{9}{4{\lambda }^2}+\frac{4}{{\lambda }^2}=25\Rightarrow \frac{9}{4{\lambda }^2}+\frac{16}{4{\lambda }^2}=\frac{25}{4{\lambda }^2}=25\Rightarrow {\lambda }^2=\frac{1}{4},\lambda =\pm \frac{1}{2}.$ Take $\lambda =+\frac{1}{2}$: $x=3,y=4$. Then $f=3(3)+4(4)=9+16=25$. The other sign gives $(-3,-4)$, $f=-25$ (the min). Max $=25$ at $(3,4)$. Sanity: $(3,4)$ is the direction of $(3,4)$ itself, scaled onto the radius-5 circle — the plane climbs fastest exactly along $\nabla f$, so the highest point is where the radius points the same way. ✔

Minimize $f=x^2+y^2+z^2$ (squared distance — easier than the square root, same minimizer) subject to $g=2x+y+2z-9=0$. $\nabla f=(2x,2y,2z)=\lambda (2,1,2)=\nabla g\cdot \lambda .$ Componentwise: $2x=2\lambda \Rightarrow x=\lambda$; $2y=\lambda \Rightarrow y=\lambda /2$; $2z=2\lambda \Rightarrow z=\lambda$. Substitute into the plane: $2\lambda +\frac{\lambda }{2}+2\lambda =9\Rightarrow \frac{9\lambda }{2}=9\Rightarrow \lambda =2$. So $(x,y,z)=(2,1,2)$, and $f^{\ast }=4+1+4=9$. Minimum squared distance $=9$ (distance $3$). Why it's a min not a max: on a flat plane, distance to origin has no upper bound but a single lowest point — the foot of the perpendicular.

$\nabla f=(2x,4y)$, $\nabla g=(2x,2y)$. Condition $\nabla f=\lambda \nabla g$: $2x=\lambda 2x,4y=\lambda 2y.$ Case A — $x\ne 0$: the first equation gives $\lambda =1$. Then the second becomes $4y=2y\Rightarrow 2y=0\Rightarrow y=0$. Constraint: $x^2=1$, so $x=\pm 1$. Points $(\pm 1,0)$, $f=1$. Case B — $y\ne 0$: the second gives $\lambda =2$. Then the first becomes $2x=4x\Rightarrow x=0$. Constraint: $y^2=1$, $y=\pm 1$. Points $(0,\pm 1)$, $f=2$. Case C — $x=0$ and $y=0$: impossible, that point is not on the circle. So no degenerate solution is lost. Classify: minimum $f=1$ at $(\pm 1,0)$ (red dots in figure), maximum $f=2$ at $(0,\pm 1)$ (mint dots). These sit exactly where the ellipse-shaped level curves of $f$ kiss the circle (Level sets and contours) — tangency is the geometric face of $\nabla f\parallel \nabla g$. Why splitting on the sign of $x,y$ matters: if you had blindly cancelled $x$ from $2x=2\lambda x$ you'd have lost the $\lambda =2$ branch (which requires $x=0$). Handling each factor being zero is the analysis skill of L3.

From the parent, at the optimum $x=y=c/2$ and $\lambda =2x=c$. Differentiate the optimal value: $\frac{d{f}^{\ast }}{dc}=\frac{d}{dc}\left(\frac{c^2}{2}\right)=c$. Indeed $\lambda =c$. ✔ Interpretation: as $c$ grows the line moves farther from the origin, so the minimum squared distance rises — a positive $\lambda$ means "relaxing the constraint outward makes $f^{\ast }$ larger." Big $c$ → steep sensitivity. This is the shadow-price statement made concrete.

$\nabla f=(0,0,1)$, $\nabla g=(2x,2y,-1)$, $\nabla h=(1,1,1)$. Condition $\nabla f=\lambda \nabla g+\mu \nabla h$: $0=2\lambda x+\mu ,0=2\lambda y+\mu ,1=-\lambda +\mu .$ Subtract the first two: $2\lambda (x-y)=0$. Case $\lambda \ne 0$: $x=y$. (Case $\lambda =0$ forces $\mu =0$ from eqn 1 yet $\mu =1$ from eqn 3 — contradiction, so discard.) Constraints with $x=y$: $z=2x^2$ and $2x+z=3$, hence $2x+2x^2=3$, i.e. $2x^2+2x-3=0\Rightarrow x=\frac{-1\pm \sqrt{7}}{2}.$ Numerically $x\approx 0.823$ or $x\approx -1.823$. Then $z=3-2x$:

• $x\approx 0.823\Rightarrow z\approx 1.354$ — the minimum of $z$.
• $x\approx -1.823\Rightarrow z\approx 6.646$ — the maximum of $z$. Why two answers: the intersection is a closed loop that dips low then rises high (see figure); $z$ has one lowest and one highest point on it. Both satisfy Lagrange because both are places $\nabla f$ sits in $\mathrm{span}\{\nabla g,\nabla h\}$. Multipliers at the min ($x=y\approx 0.823$): from $0=2\lambda x+\mu$ and $1=-\lambda +\mu$, $\mu =-2\lambda x,1=-\lambda -2\lambda x=-\lambda (1+2x)\Rightarrow \lambda =\frac{-1}{1+2x}.$ With $2x=-1+\sqrt{7}$, $1+2x=\sqrt{7}$, so $\lambda =-1/\sqrt{7}\approx -0.378$ and $\mu =-2\lambda x=2x/\sqrt{7}\cdot ?$ — compute: $\mu =-2\lambda x=\frac{2x}{1+2x}=\frac{-1+\sqrt{7}}{\sqrt{7}}\approx 0.622$.

$g=x+y+z-3$. $\nabla f=(yz,xz,xy)=\lambda (1,1,1)$. So $yz=xz=xy=\lambda$. From $yz=xz$: $z(y-x)=0$; since $z>0$, $y=x$. From $xz=xy$: $x(z-y)=0$; since $x>0$, $z=y$. Hence $x=y=z$. Constraint: $3x=3\Rightarrow x=1$. So $(1,1,1)$, $f=1\cdot 1\cdot 1=1$. Interpretation: the product of positive numbers with fixed sum is largest when they're equal — this is the AM–GM inequality falling out of Lagrange. Maximum $=1$.

Let $A=\left(\begin{array}{cc}a& b\\ b& c\end{array}\right)$ so $f={\mathbf{x}}^{\top }A\mathbf{x}$ where $\mathbf{x}=(x,y)$. Then $\nabla f=2A\mathbf{x}$ and, with $g={\mathbf{x}}^{\top }\mathbf{x}-1$, $\nabla g=2\mathbf{x}$. Lagrange demands $2A\mathbf{x}=\lambda 2\mathbf{x}$, i.e. $\boxed{A\mathbf{x}=\lambda \mathbf{x}.}$ This is exactly the eigenvalue equation: the constrained critical directions are the eigenvectors of $A$, and $\lambda$ is the eigenvalue. Value at such a point: $f={\mathbf{x}}^{\top }A\mathbf{x}={\mathbf{x}}^{\top }(\lambda \mathbf{x})=\lambda {\mathbf{x}}^{\top }\mathbf{x}=\lambda \cdot 1=\lambda$. So on the unit circle $f$ equals the eigenvalue. The maximum of $f$ is therefore the largest eigenvalue ${\lambda }_{\max }$, achieved at its eigenvector; the minimum is ${\lambda }_{\min }$. ∎ Why this is deep: the shadow-price meaning ($\lambda =$ value of $f$) becomes literal, and Lagrange has just derived the variational characterization of eigenvalues.

$\nabla f=(1,0)$, $\nabla g=(-3x^2,2y)$. At $(0,0)$: $\nabla g=(0,0)=0$. Lagrange asks for $\nabla f=\lambda \nabla g$, i.e. $(1,0)=\lambda (0,0)=(0,0)$ — impossible for any $\lambda$. So the algebra produces no solution at the cusp, even though the curve genuinely has its leftmost point (minimum of $x$) there at $x=0$. Conclusion: Lagrange misses this extremum because its core assumption — $\nabla g\ne 0$, so the surface has a well-defined normal — fails at a singular point. Fix in practice: always separately inspect points where $\nabla g=0$ and any boundaries, as KKT conditions and the parent Mistake C warn.