KKT conditions for constrained optimization: Question bank

Two reminders before we start, so every symbol below is already grounded:

True or false — justify

TF1. If a KKT point exists, it must be the global minimum.

False — for non-convex problems KKT are only necessary, so a KKT point can be a maximum, a saddle, or a local (not global) min; only convexity upgrades them to sufficient.

TF2. Equality multipliers

μ_{j}

must be non-negative.

False — only inequality multipliers

λ_{i} \geq 0

; equality multipliers

μ_{j}

are free in sign, exactly like ordinary Lagrange Multipliers.

TF3. At any feasible point, complementary slackness

λ_{i} g_{i} (x^{*}) = 0

holds automatically.

False — it is a condition KKT imposes at the optimum, not a fact about every feasible point; a random feasible point can have

g_{i} < 0

with a nonzero

λ_{i}

and violate it.

TF4. If

g_{i} (x^{*}) < 0

(fence not touched), then

λ_{i} = 0

True — slackness

λ_{i} g_{i} = 0

with

g_{i} \neq = 0

forces

λ_{i} = 0

; an untouched fence exerts no push.

TF5. If

λ_{i} > 0

, the corresponding constraint must be active.

True — slackness forces

g_{i} (x^{*}) = 0

whenever

λ_{i} \neq = 0

; a fence only pushes back when you're leaning on it.

TF6. For a convex problem, satisfying all four KKT conditions guarantees a global minimum.

True — with

f, g_{i}

convex and

h_{j}

affine, KKT are necessary and sufficient for a global min.

TF7. Every point where

\nabla f (x^{*}) = 0

(a flat spot of

f

) is a KKT point.

False — it's only KKT if it's also feasible and the multipliers can be chosen to satisfy stationarity/slackness; a flat spot outside the feasible region isn't a solution at all.

TF8. LICQ is needed for KKT to be sufficient.

False — LICQ is needed for KKT to be necessary at a minimum; sufficiency comes from convexity, a separate ingredient.

TF9. If both fences

g_{1}, g_{2}

are inactive at

x^{*}

, the problem locally behaves like an unconstrained one.

True — inactive constraints have zero multipliers and drop out of stationarity, so locally you're just at a flat spot of

f

TF10. In a convex problem, KKT points are unique.

False — convexity guarantees a global optimum value but the minimizer set can be a whole flat region (e.g. minimizing a constant on a segment), giving many KKT points.

Why questions

WHY1. Why must

λ_{i} \geq 0

but

μ_{j}

can be any sign?

A fence blocks motion only on one side (

g \leq 0

), so its push must point one way (non-negative multiplier); a wire (

h = 0

) blocks motion on both sides, so its multiplier can pull either direction.

WHY2. Why is complementary slackness the single statement that unifies "active vs inactive"?

The product

λ_{i} g_{i} = 0

forces at least one factor to vanish — either

g_{i} = 0

(active) or

λ_{i} = 0

(inactive) — capturing both cases in one equation.

WHY3. Why does KKT need a constraint qualification at all?

Without one, the active constraint gradients can be degenerate (e.g. fences meeting in a cusp), and then no multipliers exist to express

- \nabla f

as their combination even at a true minimum.

WHY4. Why does convexity turn KKT from necessary into sufficient?

In a convex problem a stationary feasible point with the right sign conditions has no feasible descent direction anywhere (not just locally), so it must be the global minimum — see Convex Optimization.

WHY5. Why, geometrically, must

- \nabla f

lie inside the cone of active constraint gradients?

- \nabla f

(the downhill pull) stuck outside that cone, some feasible direction would still reduce

f

; being trapped inside the cone means every escape route violates a constraint.

WHY6. Why does an inactive constraint contribute nothing to stationarity?

Its multiplier is forced to zero by slackness, so its gradient term drops from

\nabla_{x} L

— a fence you're not touching exerts no force.

WHY7. Why do KKT conditions matter so much for Support Vector Machines?

The SVM is a convex program, so KKT are sufficient; complementary slackness picks out exactly the support vectors (active constraints with

λ_{i} > 0

) that define the margin.

WHY8. Why does duality relate to KKT?

KKT stationarity is exactly

\nabla_{x} L = 0

, and

λ_{i} \geq 0

with slackness are the conditions under which the primal and dual optima coincide (strong duality).

Edge cases

EC1. What are the KKT conditions when there are no inequality constraints, only equalities?

They collapse to ordinary Lagrange Multipliers:

\nabla f + \sum_{j} μ_{j} \nabla h_{j} = 0

with feasibility

h_{j} = 0

, and no sign restriction on any multiplier.

EC2. What happens to KKT if there are no constraints at all?

Stationarity reduces to

\nabla f (x^{*}) = 0

— the plain unconstrained first-order condition, the "flat ground" case.

EC3. A constraint is active and its multiplier is exactly zero (

g_{i} = 0

λ_{i} = 0

). Is that allowed?

Yes — this "degenerate" or weakly active case satisfies slackness; the fence is touched but exerts no push, often a sign the optimum sits right at a boundary that isn't binding.

EC4. The feasible region is empty (constraints contradict). What do KKT say?

Nothing — KKT presupposes a feasible optimum; with no feasible point there is no

x^{*}

to test, so the whole framework doesn't apply.

EC5. The objective is unbounded below on the feasible set. Can a KKT point still exist?

No finite minimizer exists, so there's no KKT point; stationarity would require a balance that never occurs as

f \to - \infty

EC6. Two fences

g_{1}, g_{2}

have identical gradients at

x^{*}

(linearly dependent). What breaks?

LICQ fails, so the multipliers

λ_{1}, λ_{2}

may be non-unique or non-existent — you can't cleanly read off the forces even if the point is optimal.

EC7. Can a KKT point be a local maximum of

f

Yes, for non-convex problems — stationarity

\nabla_{x} L = 0

is symmetric to min/max, so KKT catches maxima and saddles too unless convexity rules them out.

EC8. What does it mean if projected gradient descent stalls at a point where the projected gradient is zero?

It has reached a KKT point of the constrained problem — the descent direction is fully cancelled by the constraint projection, mirroring stationarity plus slackness.

Recall One-line summary of the deepest trap

The two conditions people confuse: $λ_{i} \geq 0$ is about sign (fences push one way), slackness $λ_{i} g_{i} = 0$ is about activity (fences push only when touched) — they're different rules, both required.

Question bank — KKT conditions for constrained optimization

True or false — justify

Spot the error

Why questions

Edge cases