Exercises — Gradient as direction of steepest ascent

WHAT: we just need the two partial derivatives, then plug in the point. WHY partials: the gradient is by definition the list of how fast $f$ changes if you nudge one variable while freezing the other. $f_x=\frac{\partial }{\partial x}(3x^2-5y)=6x,f_y=\frac{\partial }{\partial y}(3x^2-5y)=-5.$ Plug in $(2,1)$: $\nabla f(2,1)=(6\cdot 2,-5)===(12,-5)==$. Notice $f_y$ has no $x$ or $y$ left in it — that is fine, a partial can be a constant.

WHAT: identify which candidate has length $1$. WHY it matters: the direction of steepest ascent is the gradient divided by its own length — a unit arrow. The raw gradient $(2,4)$ has length $\sqrt{4+16}=\sqrt{20}\ne 1$, so it is a direction and a rate glued together, not a pure direction. Check $\frac{1}{\sqrt{5}}(1,2)$: length $=\frac{1}{\sqrt{5}}\sqrt{1+4}=\frac{\sqrt{5}}{\sqrt{5}}=1$. ✓ Answer: $==\frac{1}{\sqrt{5}}(1,2)==$.

True. A level curve is the set of points where $f$ keeps the same height (see Level Curves and Contour Maps). If height does not change as you walk along it, the rate of change $D_{\mathbf{u}}f$ is exactly $0$.

(a) Partials. Treat the other variable as a frozen constant. $f_x=2xy,f_y=x^2\Rightarrow \nabla f(2,3)=(2\cdot 2\cdot 3,2^2)=(12,4).$ (b) Normalise. Length $\Vert \nabla f\Vert =\sqrt{12^2+4^2}=\sqrt{144+16}=\sqrt{160}=4\sqrt{10}$. ${\mathbf{u}}^{\ast }=\frac{(12,4)}{4\sqrt{10}}=\frac{(3,1)}{\sqrt{10}}.$ (c) Max rate $=\Vert \nabla f\Vert =4\sqrt{10}\approx 12.649$. Why: the biggest possible $\nabla f\cdot \mathbf{u}$ is $\Vert \nabla f\Vert \cos 0=\Vert \nabla f\Vert$.

Step 1 — direction vector = tip minus tail: $(4-1,6-2)=(3,4)$. Step 2 — normalise (else the rate is scaled by the wrong length): $\Vert (3,4)\Vert =5$, so $\mathbf{u}=(3,4)/5$. Step 3 — dot with the gradient $\nabla f(1,2)=(2,4)$: $D_{\mathbf{u}}f=(2,4)\cdot \left(\frac{3}{5},\frac{4}{5}\right)=\frac{6+16}{5}=\frac{22}{5}===4.4==.$ Sanity check: $\Vert \nabla f\Vert =2\sqrt{5}\approx 4.472>4.4$, so we're below the maximum — correct, nothing beats the gradient direction.

WHY $e^x$ stays and $\cos y$ becomes $-\sin y$: partial $\partial /\partial x$ freezes $y$, so $\cos y$ is a constant multiplier; partial $\partial /\partial y$ freezes $x$, so $e^x$ is a constant multiplier. $f_x=e^x\cos y,f_y=-e^x\sin y.$ At $(0,0)$: $e^0=1$, $\cos 0=1$, $\sin 0=0$, so $\nabla f=(1\cdot 1,-1\cdot 0)=(1,0)$. Max rate $=\Vert (1,0)\Vert ===1==$.

Step 1 — gradient: $f_x=y,f_y=x$, so $\nabla f(2,3)=(3,2)$. Step 2 — tangent to the curve. Differentiate $xy=6$ implicitly (treat $y$ as a function of $x$): $y+x{y}^{\prime }=0\Rightarrow y^{\prime }=-\frac{y}{x}=-\frac{3}{2}$. A slope of $-\frac{3}{2}$ means a tangent direction $\propto (2,-3)$ (run $2$, rise $-3$). Step 3 — dot product test for perpendicularity (two vectors are $\perp$ exactly when their dot product is $0$, from Dot Product and Angle): $\nabla f\cdot (2,-3)=(3)(2)+(2)(-3)=6-6===0==.✓$ So the gradient is perpendicular to the level curve, exactly as the theory predicts.

WHY the geometric form: we know the angle, not the components of $\mathbf{u}$, so use $D_{\mathbf{u}}f=\Vert \nabla f\Vert \cos \theta$ directly. $\Vert \nabla f\Vert =\sqrt{9+16}=5$, $\cos 60^{\circ }=\frac{1}{2}$. $D_{\mathbf{u}}f=5\times \frac{1}{2}===2.5==.$ Interpretation: you're getting half the maximum steepness because you're walking $60^{\circ }$ off the straight-uphill line.

$f_x=2x,f_y=-2y$, so $\nabla f(0,0)=(0,0)$ — the zero vector. WHAT it means: $\Vert \nabla f\Vert =0$, so the maximum rate of increase is $0$. Every direction is flat to first order — this is a critical point (here a saddle). There is no unique steepest-ascent direction, because $\nabla f/\Vert \nabla f\Vert$ divides by zero and is undefined. This is the degenerate case: the steepest-ascent recipe only names a direction when $\nabla f\ne 0$.

Gradient: $T_x=-2x,T_y=-4y$, so $\nabla T(3,1)=(-6,-4)$. (a) Warmest-fastest is the gradient direction, normalised. $\Vert \nabla T\Vert =\sqrt{36+16}=\sqrt{52}=2\sqrt{13}$. ${\mathbf{u}}^{\ast }=\frac{(-6,-4)}{2\sqrt{13}}=\frac{(-3,-2)}{\sqrt{13}}.$ The bug heads down-and-to-the-left (toward the origin, where it's warmest — makes sense, $T$ peaks at $(0,0)$). (b) Rate $=\Vert \nabla T\Vert =2\sqrt{13}\approx ==7.211==$. (c) Due east $\mathbf{u}=(1,0)$ is already a unit vector: $D_{\mathbf{u}}T=(-6,-4)\cdot (1,0)===-6==$. Moving east it cools down by $6$ per unit distance.

Gradient: $f_x=-2x,f_y=-2y$, so $\nabla f(1,1)=(-2,-2)$. (a) Steepest descent = $-\nabla f$, then normalise: $-\nabla f=(2,2)$, length $2\sqrt{2}$, so unit direction $=\frac{(2,2)}{2\sqrt{2}}=\frac{(1,1)}{\sqrt{2}}$. (b) Gradient descent uses the raw gradient (not normalised) in its update: ${\mathbf{x}}_{\text{new}}=(1,1)-0.1\cdot (-2,-2)=(1,1)+(0.2,0.2)===(1.2,1.2)==.$ Note: descent subtracts the gradient, and since $\nabla f$ points uphill, $-\eta \nabla f$ nudges us toward the peak of this dome — consistent with $f$ increasing toward $(0,0)$... wait: here $f$ peaks at the origin, and we moved away from it to $(1.2,1.2)$. That means we are climbing the outer bowl? No — recheck: $-\eta \nabla f=+(0.2,0.2)$ moves us to larger $x,y$, where $f=9-x^2-y^2$ is smaller. So $f$ decreased: gradient descent correctly reduced $f$. ✓

WHY the chain rule inside: $f$ is $\ln$ of an inner function $g=x^2+y^2$; each partial multiplies $\frac{1}{g}$ by the partial of $g$ (see Multivariable Chain Rule). $f_x=\frac{2x}{x^2+y^2},f_y=\frac{2y}{x^2+y^2}.$ At $(1,2)$: $x^2+y^2=5$, so $\nabla f=\left(\frac{2}{5},\frac{4}{5}\right)$. Max rate $=\Vert \nabla f\Vert =\sqrt{(2/5{)}^2+(4/5{)}^2}=\sqrt{\frac{4+16}{25}}=\sqrt{\frac{20}{25}}=\frac{2}{\sqrt{5}}\approx ==0.894==$.

Steepest ascent is undefined exactly where $\nabla f=0$ (can't normalise the zero vector, per L3·Q3). $f_x=2x+y=0,f_y=x+2y=0.$ Solve: from the first, $y=-2x$; substitute into the second: $x+2(-2x)=x-4x=-3x=0\Rightarrow x=0$, hence $y=0$. Only point: $==(0,0)==$. Since $f=x^2+xy+y^2=(x+\frac{y}{2}{)}^2+\frac{3}{4}{y}^2\ge 0$ with equality only at the origin, it's a minimum. So there's exactly one point with no steepest-ascent direction, and it's the bottom of the bowl.

WHY this reproves the parent's key claim — but with single-variable optimisation instead of the dot-product inequality. $D(\varphi )=\nabla f\cdot (\cos \varphi ,\sin \varphi )=2\cos \varphi +0\cdot \sin \varphi =2\cos \varphi$. Differentiate: $D^{\prime }(\varphi )=-2\sin \varphi$. Set $=0\Rightarrow \sin \varphi =0\Rightarrow \varphi =0$ or $\pi$. Second derivative: $D^{\prime \prime }(\varphi )=-2\cos \varphi$. At $\varphi =0$: $D^{\prime \prime }(0)=-2<0$ → maximum. At $\varphi =\pi$: $D^{\prime \prime }(\pi )=+2>0$ → minimum (that's steepest descent). Maximum value $D(0)=2\cos 0===2===\Vert \nabla f\Vert =\Vert (2,0)\Vert$. ✓ The gradient's magnitude is the peak rate.

(a) $D_{\mathbf{u}}f=(a,b)\cdot \frac{1}{\sqrt{2}}(1,-1)=\frac{a-b}{\sqrt{2}}.$ (b) "Along a level curve" means the height isn't changing: $D_{\mathbf{u}}f=0\Rightarrow \frac{a-b}{\sqrt{2}}=0\Rightarrow ==a=b==$. Geometric read: when $a=b$ the gradient $(a,a)$ points along $(1,1)$, which is exactly perpendicular to $\mathbf{u}\propto (1,-1)$ — so the walker is skirting the hill at constant height.