Optimization — constrained, unconstrained, real-world problems

WHY does the derivative vanish at an extremum?

Deriving the classification test from scratch

We want a way to tell a peak from a valley. Use the Taylor expansion near a critical point $c$ (where $f'(c)=0$):

$f(c+h) = f(c) + f'(c)h + \tfrac{1}{2}f''(c)h^2 + \cdots$

Since $f'(c)=0$, the leading change is

$f(c+h) - f(c) \approx \tfrac{1}{2}f''(c)\,h^2.$

Why this step? $h^2>0$ for any small $h\neq 0$, so the sign of $f''(c)$ alone decides whether $f$ goes up or down on both sides.

Unconstrained vs Constrained — WHAT is the difference?

Why Lagrange multipliers? (geometric derivation)

At a constrained optimum we move along the curve $g=0$. We can't increase $f$ while staying on the curve only when $f$ stops changing along the curve — i.e. $\nabla f$ has no component tangent to $g=0$. That means $\nabla f$ is parallel to $\nabla g$:

$\boxed{\nabla f = \lambda \nabla g}, \qquad g(x,y)=0.$

Why this step? $\nabla g$ is perpendicular to the constraint curve; if $\nabla f$ is also perpendicular, no tangent direction can improve $f$. The scalar $\lambda$ measures how much the optimum would change if the constraint loosened.

The universal recipe (the 80/20 core)

Worked Example 1 — Box of maximum volume (constrained → 1-D)

A 12 cm × 12 cm sheet has equal squares of side $x$ cut from each corner; flaps fold up into an open box. Maximize volume.

• Dimensions after folding: base $(12-2x)$, height $x$.
• $V(x) = x(12-2x)^2$, valid for $0 \le x \le 6$. Why? $x<0$ meaningless, $x>6$ gives negative base.

Differentiate (product rule): $V'(x) = (12-2x)^2 + x\cdot 2(12-2x)(-2) = (12-2x)\big[(12-2x) - 4x\big].$ Why factor? It instantly reveals roots. $V'(x) = (12-2x)(12-6x) = 0 \Rightarrow x = 6 \text{ or } x = 2.$

Classify by endpoints: $V(0)=0,\ V(6)=0,\ V(2)=2(8)^2=128$. So $x=2$ gives maximum volume $128\ \text{cm}^3$.

Worked Example 2 — Closest point (substitution)

Find the point on the line $y = 2x + 1$ nearest the origin.

Minimize distance, but minimize $D^2$ instead of $D$. Why? $\sqrt{\cdot}$ is increasing, so the minimizer is the same and algebra is cleaner. $S(x) = x^2 + y^2 = x^2 + (2x+1)^2 = 5x^2 + 4x + 1.$ $S'(x) = 10x + 4 = 0 \Rightarrow x = -\tfrac{2}{5}.$ $S''(x) = 10 > 0$ → minimum. Then $y = 2(-\tfrac25)+1 = \tfrac15$.

Nearest point: $\left(-\tfrac25,\ \tfrac15\right)$, distance $=\sqrt{S}= \sqrt{\tfrac15}=\tfrac{1}{\sqrt5}$. Sanity check: the segment to this point should be perpendicular to the line (slope $-\tfrac12$ vs line slope $2$, product $=-1$ ✓).

Worked Example 3 — Lagrange multipliers

Maximize $f(x,y)=xy$ subject to $x+y=10$.

Set $\nabla f = \lambda\nabla g$ with $g = x+y-10$: $y = \lambda,\quad x = \lambda \Rightarrow x = y.$ Why? Both partials of $g$ are $1$, so $f_x=f_y \Rightarrow y=x$. Constraint gives $2x=10 \Rightarrow x=y=5$, so max product $=25$.

This recovers the famous result: among numbers with fixed sum, the product is largest when they are equal (AM–GM in disguise).

Flashcards

Connections

• Critical Points & Fermat's Theorem
• Second Derivative & Concavity
• Taylor Series Expansion
• Closed Interval Method (Global Extrema)
• Lagrange Multipliers (Multivariable)
• AM-GM Inequality
• Related Rates — both translate words into derivative equations.

Concept Map

Hinglish (regional understanding)