Exercises — First-order optimality conditions — gradient = 0

Kya karein: wahan dhundho jahan slope flat ho. Kyun: ek interior optimum $f^{\prime }=0$ force karta hai. $f^{\prime }(x)=2x-6$. Set karo $2x-6=0\Rightarrow x=3$. Stationary point hai $x=3$ (aur $f(3)=9-18+5=-4$).

(a) $f^{\prime }=2x$, $0$ par deta hai $0$ — haan (ek minimum). (b) $f^{\prime }=3x^2$, $0$ par deta hai $0$ — haan (lekin ek inflection, extremum nahi). (c) $f^{\prime }=-\sin x$, $0$ par deta hai $0$ — haan (ek maximum, kyunki $\cos 0=1$ peak hai). (d) $\nabla f=(2x,-2y)$, $(0,0)$ par deta hai $0$ — haan (ek saddle). Chaaron stationary hain; sirf kuch extrema hain — bilkul wahi FONC ki warning hai.

Galat. $f^{\prime }(a)=0$ interior optimum ke liye zaroori hai, sufficient nahi. Counterexample $f(x)=x^3$ at $a=0$: flat hai lekin na max na min.

$f^{\prime }(x)=3x^2-6x-9=3(x^2-2x-3)=3(x-3)(x+1)$. Set $=0\Rightarrow x=3$ ya $x=-1$. Do stationary points: $x=-1$ aur $x=3$.

$\nabla f=(2x-4,2y+6)$. Har ek ko zero set karo: $2x-4=0\Rightarrow x=2$; $2y+6=0\Rightarrow y=-3$. Stationary point $(2,-3)$, jahan $f(2,-3)=4+9-8-18+13=0$.

$\nabla f=(2x+y,x+2y-3)$. System solve karo: $2x+y=0,x+2y-3=0.$ Pehle se, $y=-2x$. Substitute karo: $x+2(-2x)-3=0\Rightarrow -3x-3=0\Rightarrow x=-1$, toh $y=2$. Stationary point $(-1,2)$; $f(-1,2)=1-2+4-6=-3$. ✓

$f^{\prime \prime }(x)=6x-6$. $x=-1$ par: $f^{\prime \prime }=-12<0$ ⟹ local maximum, value $f(-1)=-1-3+9+2=7$. $x=3$ par: $f^{\prime \prime }=12>0$ ⟹ local minimum, value $f(3)=27-27-27+2=-25$.

$f_{xx}=2,f_{yy}=2,f_{xy}=1$. Toh $D=2\cdot 2-1^2=3>0$ aur $f_{xx}=2>0$. ⟹ local minimum (actually global — function convex hai). Value $-3$.

$\nabla f=(3x^2-3,2y)$. Set $=0$: $3x^2-3=0\Rightarrow x=\pm 1$; $2y=0\Rightarrow y=0$. Do candidates: $(1,0)$ aur $(-1,0)$. Second slopes: $f_{xx}=6x,f_{yy}=2,f_{xy}=0$, toh $D=12x$.

• $(1,0)$ par: $D=12>0,f_{xx}=6>0$ ⟹ minimum, value $f=1-3+0=-2$.
• $(-1,0)$ par: $D=-12<0$ ⟹ saddle, value $f=-1+3+0=2$.

Errors: teen points par $m{x}_i+b-y_i$ equals $b-1$, $m+b-3$, $2m+b-2$. $S=(b-1{)}^2+(m+b-3{)}^2+(2m+b-2{)}^2$. Kyun FONC: is smooth bowl ka minimum interior hai, toh $\nabla S=0$. $\frac{\partial S}{\partial m}=2(m+b-3)+2\cdot 2(2m+b-2)=2(5m+3b-7)=0,$ $\frac{\partial S}{\partial b}=2(b-1)+2(m+b-3)+2(2m+b-2)=2(3m+3b-6)=0.$ System: $5m+3b=7$ aur $3m+3b=6$. Subtract karo: $2m=1\Rightarrow m=\frac{1}{2}$. Phir $3(\frac{1}{2})+3b=6\Rightarrow 3b=4.5\Rightarrow b=1.5$. Best-fit line $y=\frac{1}{2}x+\frac{3}{2}$. (Yahi bilkul least-squares normal-equation logic hai.)

(a) Substitute karo $y=1-x$: $g(x)=x^2+(1-x{)}^2=2x^2-2x+1$. $g^{\prime }(x)=4x-2=0\Rightarrow x=\frac{1}{2}$, toh $y=\frac{1}{2}$, $f=\frac{1}{4}+\frac{1}{4}=\frac{1}{2}$. (b) Lagrangian $L=x^2+y^2-\lambda (x+y-1)$. $L$ par FONC: $2x-\lambda =0,2y-\lambda =0\Rightarrow x=y$. $x+y=1$ ke saath: $x=y=\frac{1}{2}$. Same answer $f=\frac{1}{2}$. Boundary gradient yahan vanish nahi hota — constraint $\lambda$ bacha hua slope absorb karta hai, aur yahi reason hai kyun plain FONC constrained problems par fail karta hai.

$\nabla f=(2x,6y)$; $(2,1)$ par woh hai $(4,6)$. Descent rule: $x_{\text{new}}=x_{\text{old}}-\eta \nabla f$. $x_{\text{new}}=2-0.1\cdot 4=1.6$, $y_{\text{new}}=1-0.1\cdot 6=0.4$. Naya point $(1.6,0.4)$. Stationary point hai $(0,0)$ jahan $\nabla f=0$; distance $\sqrt{5}\approx 2.236$ se gir kar $\sqrt{1.6^2+0.4^2}=\sqrt{2.72}\approx 1.649$ ho gaya. Karib. ✓

Local max: $\exists \delta >0$ jahan $f(x^{\ast }+h)\le f(x^{\ast })$ for $|h|<\delta$, toh numerator $f(x^{\ast }+h)-f(x^{\ast })\le 0$.

• Right, $h\to 0^{+}$: numerator $\le 0$, denominator $>0$ ⟹ quotient $\le 0$ ⟹ $f^{\prime }(x^{\ast })\le 0$.
• Left, $h\to 0^{-}$: numerator $\le 0$, denominator $<0$ ⟹ quotient $\ge 0$ ⟹ $f^{\prime }(x^{\ast })\ge 0$. Dono ⟹ $f^{\prime }(x^{\ast })=0$. Differentiability dono one-sided limits ko equal banati hai, aur humne us common value ko $0$ par trap kar liya. $■$

Yaad karo $D_{u}f(x^{\ast })=\nabla f(x^{\ast })\cdot u$ (directional derivative). Lo $u=+\nabla f/\Vert \nabla f\Vert$. Toh $D_{u}f=\Vert \nabla f\Vert >0$, toh $f$ $u$ ke along increase karta hai — maximum ko beat karta hai. Lo $u=-\nabla f/\Vert \nabla f\Vert$. Toh $D_{u}f=-\Vert \nabla f\Vert <0$, toh $f$ decrease karta hai — minimum ko beat karta hai. Kyunki hum dono max sense aur min sense mein strictly improve kar sakte hain, $x^{\ast }$ na to max hai na min. Contrapositive: kisi bhi interior extremum par $\nabla f=0$ hoga. $■$

$\nabla f=(4x^3,4y^3)=0$ sirf $(0,0)$ par. Second slopes: $f_{xx}=12x^2,f_{yy}=12y^2,f_{xy}=0$; origin par sab zero hain, toh $D=0$ — test fail karta hai. Lekin seedha dekho $f(x,y)=x^4+y^4\ge 0=f(0,0)$ sab $(x,y)$ ke liye, equality sirf origin par. Toh $(0,0)$ strict global minimum hai. Moral: jab $D=0$ ho, function khud inspect karo.

Maano $x^{\ast }$ stationary hai, $\nabla f(x^{\ast })=0$. Convexity inequality mein $x=x^{\ast }$ daalo: $f(y)\ge f(x^{\ast })+\underset{=0}{\underbrace{\nabla f(x^{\ast })}}\cdot (y-x^{\ast })=f(x^{\ast })\text{for all }y.$ Isliye $f(y)\ge f(x^{\ast })$ har jagah — ek global minimum. Yahi reason hai kyun convexity mere "stationary" wali shortlist ko guaranteed winner mein convert karti hai. $■$

FONC ($\nabla f=0$) ek net hai jo har interior peak, valley, aur saddle ko pakadta hai. Levels 1–2 ne tumhe net cast karna sikhaya (system solve karo). Level 3 ne catch sort karna sikhaya (Hessian $D$). Level 4 ne un machhliyon ko handle kiya jo deewaroon ke paas tairti hain (constraints, algorithms). Level 5 ne prove kiya kyun net kaam karta hai aur sort kab fail hota hai. Flat hona finalist hai — Hessian, convexity, ya direct inspection winner tay karta hai.