Worked examples — KKT conditions for constrained optimization

Before anything, let us re-anchor the vocabulary so nothing is used unexplained.

Recall the working recipe (we use it in every example):

1. Write $\mathcal{L}=f+{\sum }_i{\lambda }_i{g}_i+{\sum }_j{\mu }_j{h}_j$ (put every $g$ in $\le 0$ form first!).
2. Set ${\nabla }_x\mathcal{L}=0$ (stationarity).
3. Guess which inequalities are active. Inactive ⇒ its $\lambda =0$; active ⇒ its $g=0$.
4. Solve. Then verify: primal feasibility, $\lambda \ge 0$, slackness. If any $\lambda <0$, the guess was wrong — try another active-set.

---

The scenario matrix

#	Cell (what makes it different)	Example
A	All inequalities inactive (interior min)	Ex 1
B	One inequality active, positive $\lambda$	Ex 2
C	Equality + inequality together, $\mu <0$ (sign-free)	Ex 3
D	Wrong active-set guess → $\lambda <0$ flag, then repair	Ex 4
E	Two inequalities active at once (a cone / corner)	Ex 5
F	Degenerate LICQ failure — multipliers don't exist	Ex 6
G	Limiting case: constraint budget $\to$ moves the answer smoothly	Ex 7
H	Word problem (real units) with an active budget	Ex 8
I	Exam twist: non-convex, KKT catches a maximum too	Ex 9

The rows cover: every activity combination (inactive / one active / two active), both multiplier signs, the failure of the qualification, a limit, a real-world model, and a non-convex trap.

---

Example A — everything inactive (interior minimum)

1. Lagrangian. $\mathcal{L}=(x-2{)}^2+(y-1{)}^2+{\lambda }_1(x+y-10)+{\lambda }_2(-x)$. Why this step? Both fences are already in $\le 0$ form, so we just append them.
2. Guess both inactive ⇒ ${\lambda }_1={\lambda }_2=0$. Why this step? The cheapest possible outcome is the unconstrained minimum; test it first.
3. Stationarity. ${\partial }_x\mathcal{L}=2(x-2)=0\Rightarrow x=2$; ${\partial }_y\mathcal{L}=2(y-1)=0\Rightarrow y=1$. Why this step? $\nabla f=0$ is where the ground is flat — no fence needed.
4. Check feasibility. $g_1=2+1-10=-7\le 0$ ✓, $g_2=-2\le 0$ ✓. Slackness ${\lambda }_i{g}_i=0$ ✓.

---

Example B — one fence active (positive $\lambda$)

1. Lagrangian. $\mathcal{L}=x^2+y^2+\lambda (1-x-y)$. Why this step? Standard build; the fence is already $\le 0$.
2. Stationarity. $2x-\lambda =0,2y-\lambda =0\Rightarrow x=y=\lambda /2$. Why this step? At rest, $\nabla f=(2x,2y)$ must be cancelled by the fence's push $\lambda \nabla g=\lambda (-1,-1)$.
3. Guess active ⇒ $g=0\Rightarrow x+y=1\Rightarrow 2x=1\Rightarrow x=y=\frac{1}{2}$. Why this step? Origin is infeasible, so the answer must sit on the boundary.
4. Multiplier. $\lambda =2x=1$. Check $\lambda =1\ge 0$ ✓ (dual feasibility).

---

Example C — equality + inequality, negative $\mu$

1. Lagrangian. $\mathcal{L}=x^2+y^2+\mu (x+y-2)+\lambda (0.5-x)$. Why this step? Equality gets a free-sign $\mu$; inequality gets $\lambda \ge 0$.
2. Guess $g$ inactive ⇒ $\lambda =0$. Why this step? The wire alone might already hold the ball where $x>0.5$.
3. Stationarity. $2x+\mu =0,2y+\mu =0\Rightarrow x=y$. With $x+y=2$: $x=y=1$. Why this step? On the wire, both partials give the same $\mu$, forcing symmetry.
4. Multiplier & checks. $\mu =-2x=-2$. Check $g=0.5-1=-0.5\le 0$ ✓ (fence untouched). Slackness $\lambda g=0$ ✓. Why this step? We assumed $g$ inactive in step 2; that assumption is only legitimate if the solved point actually satisfies $g<0$ — so we must plug back and confirm, and confirm slackness $\lambda g=0$ holds automatically since $\lambda =0$.

---

Example D — wrong guess flags itself ($\lambda <0$), then repair

1. Wrong guess: $g_1$ active ⇒ $x=5$, ${\lambda }_2=0$. Why this step? To demonstrate the self-correcting flag the parent note warns about.
2. Stationarity. $\mathcal{L}=(x-3{)}^2+{\lambda }_1(x-5)+{\lambda }_2(-x)$, so $2(x-3)+{\lambda }_1-{\lambda }_2=0$. At $x=5,{\lambda }_2=0$: $2(2)+{\lambda }_1=0\Rightarrow {\lambda }_1=-4$. Why this step? Solving the assumed system gives the multiplier.
3. Read the flag. ${\lambda }_1=-4<0$ violates dual feasibility — this active-set is impossible. Discard it. Why this step? A negative inequality multiplier is the mathematical scream "this fence would have to pull, but fences only push."
4. Correct guess: both inactive. ${\lambda }_1={\lambda }_2=0\Rightarrow 2(x-3)=0\Rightarrow x=3$. Feasibility $0\le 3\le 5$ ✓, slackness ✓. Why this step? Having ruled out $g_1$ active, the only remaining candidate active-set that respects $\lambda \ge 0$ is "no fence active"; we test it and it passes all four KKT conditions, so it is the genuine optimum.

---

Example E — two fences active at once (a corner / cone)

1. Lagrangian. $\mathcal{L}=(x+2{)}^2+(y+3{)}^2+{\lambda }_1(-x)+{\lambda }_2(-y)$. Why this step? Two inequalities, both in $\le 0$ form.
2. Guess both active ⇒ $x=0,y=0$. Why this step? Both fences point toward the origin from the target; a corner is the natural rest.
3. Stationarity. ${\partial }_x:2(x+2)-{\lambda }_1=0\Rightarrow {\lambda }_1=2(0+2)=4$; ${\partial }_y:2(y+3)-{\lambda }_2=0\Rightarrow {\lambda }_2=6$. Why this step? Each active fence supplies its own push ${\lambda }_i$ to cancel that component of $\nabla f$.
4. Checks. ${\lambda }_1=4\ge 0$, ${\lambda }_2=6\ge 0$ ✓. Both $g_i=0$ so slackness ✓. Why this step? The "both active" guess is only valid if both multipliers come out non-negative (dual feasibility); we must confirm each ${\lambda }_i\ge 0$, otherwise a negative one would flag the corner as the wrong active-set.

---

Example F — degenerate: LICQ fails, no multipliers

1. Active gradients at $(0,0)$. Both constraints are active there ($g_1=g_2=0$). $\nabla g_1=(-3x^2,1)=(0,1)$ and $\nabla g_2=(-3x^2,-1)=(0,-1)$. Why this step? KKT stationarity needs $\nabla f$ to be a non-negative combination of these.
2. Stationarity attempt. $\nabla f=(1,0)$. We need $(1,0)+{\lambda }_1(0,1)+{\lambda }_2(0,-1)=0$. Why this step? Write out the required cancellation componentwise.
3. Contradiction. The $x$-component reads $1+0=0$ — impossible. No ${\lambda }_1,{\lambda }_2$ exist. Why this step? Both active gradients are vertical $(0,\pm 1)$; they are linearly dependent and span no horizontal direction, so they can never cancel $\nabla f$'s horizontal $1$.
4. Diagnosis. LICQ fails: $\nabla g_1=-\nabla g_2$ are not linearly independent (each is a scalar multiple of the other). Hence KKT are not necessary here even though $(0,0)$ is the true optimum.

---

Example G — limiting behaviour: shifting the budget

1. General solve (active). As in Example B: $x=y=c/2$, $\lambda =c$. Why this step? The geometry is identical; only the boundary line moved.
2. Multiplier meaning. $\lambda =\frac{d{f}^{\ast }}{dc}$: here $f^{\ast }=2(c/2{)}^2=c^2/2$, so $\frac{d{f}^{\ast }}{dc}=c=\lambda$. Why this step? The multiplier is the sensitivity — the marginal cost of tightening the budget by one unit (the bridge to Duality and the Dual Problem).
3. Limit $c\to 0^{+}$. $x=y\to 0$, $\lambda \to 0$: the fence becomes tangent to the free minimum and its push vanishes smoothly — a graceful hand-off from active to inactive. Why this step? This is exactly complementary slackness in a limit: $\lambda g\to 0$ from both factors shrinking.
4. Case $c<0$. Now the origin already satisfies $x+y\ge c$; the constraint is inactive, $\lambda =0$, answer $(0,0)$, $f=0$. Why this step? Negative budget is a fence behind the ball — irrelevant.

---

Example H — word problem (real units)

1. Model. Minimize $f=x^2+2y^2$ s.t. $g:6-x-y\le 0$. (Ignore $x,y\ge 0$ for now; we'll check.) Why this step? Translate "at least 6 tonnes" into $\le 0$ form.
2. Lagrangian & stationarity. $\mathcal{L}=x^2+2y^2+\lambda (6-x-y)$. ${\partial }_x:2x-\lambda =0$, ${\partial }_y:4y-\lambda =0$. Why this step? Balance marginal cost of each product against the same contract push $\lambda$.
3. Guess active ($6-x-y=0$). From $2x=\lambda =4y\Rightarrow x=2y$. Then $2y+y=6\Rightarrow y=2,x=4$. Why this step? The contract is demanding; the cheapest feasible plan sits on the boundary.
4. Multiplier & checks. $\lambda =2x=8\ge 0$ ✓. $x=4\ge 0,y=2\ge 0$ ✓. Slackness ✓. Why this step? The whole "active" guess only stands if the multiplier is non-negative and the produced amounts are physically valid (no negative tonnage); we plug the solved values back to confirm feasibility, dual feasibility, and slackness before trusting the plan.

---

Example I — exam twist: non-convex, KKT catches a maximum

1. Interior stationary point. $\mathcal{L}=-x^2+{\lambda }_1(x-1)+{\lambda }_2(-x-1)$. Guessing both inactive (${\lambda }_1={\lambda }_2=0$): $f^{\prime }(x)=-2x=0\Rightarrow x=0$. Feasibility $-1\le 0\le 1$ ✓, slackness ✓ — so $x=0$ passes all four KKT conditions. Why this step? To show that KKT can hold at a point that is not a minimum.
2. Identify $x=0$. $f(0)=0$, but nearby $f(\pm 0.1)=-0.01<0$: so $x=0$ is a local maximum of $f$, yet it passed KKT. Why this step? Because the problem is non-convex, KKT are only necessary, not sufficient — they cannot tell min from max (the parent-note mistake made concrete).
3. Boundary case $x=1$ (fence $g_1$ active). Set $g_1=0\Rightarrow x=1$, ${\lambda }_2=0$. Stationarity $-2x+{\lambda }_1-{\lambda }_2=0\Rightarrow -2(1)+{\lambda }_1=0\Rightarrow {\lambda }_1=2\ge 0$ ✓. Slackness ${\lambda }_1{g}_1=0$ ✓ (since $g_1=0$). Why this step? Test the right-hand corner as its own active-set; it must satisfy dual feasibility to count as a KKT point.
4. Boundary case $x=-1$ (fence $g_2$ active). Set $g_2=0\Rightarrow x=-1$, ${\lambda }_1=0$. Stationarity $-2(-1)+0-{\lambda }_2=0\Rightarrow 2-{\lambda }_2=0\Rightarrow {\lambda }_2=2\ge 0$ ✓. Slackness ✓. Why this step? By symmetry the left corner is the mirror case; confirm it too is a valid KKT point.
5. Decide by comparing values. KKT-satisfying points are $x\in \{-1,0,1\}$ with $f=\{-1,0,-1\}$. Pick the smallest: $x=\pm 1$, $f=-1$. Why this step? When KKT are not sufficient, they only list candidates — you must evaluate $f$ at every KKT point and choose the smallest to find the true minimum.

---

Wrap-up

Recall Which cells did we cover?

A interior-inactive · B one-active ($\lambda >0$) · C equality with $\mu <0$ · D wrong-guess flag · E two-active corner · F LICQ failure · G limiting budget · H word problem · I non-convex catches a max. Every combination of activity, both signs, degeneracy, a limit, real units, and the convex-vs-non-convex distinction.

Recall Where does the multiplier "sensitivity" idea lead next?

$\lambda =d{f}^{\ast }/dc$ is the seed of Duality and the Dual Problem and underlies why Support Vector Machines read their support vectors off active KKT constraints. Optimization method: Gradient Descent and Projected Gradient.

Back to the parent topic.