Exercises — Partial derivatives

Freeze $y$ to find $\partial f/\partial x$. The term $5x$ has slope $5$ in $x$; the term $7y$ has no $x$, so its $x$-slope is $0$. $\frac{\partial f}{\partial x}=5,\frac{\partial f}{\partial y}=7$ Each variable's coefficient is just its own slope — a flat plane tilts by a fixed amount in each direction.

Differentiate $x^2$ normally ($\to 2x$); $y^3$ has no $x$ so vanishes. $\frac{\partial f}{\partial x}=2x,\frac{\partial f}{\partial y}=3y^2$ Notice the two answers are completely decoupled — no cross terms exist, so each depends on only its own variable.

$\partial f/\partial x$: freeze $y$, so $y^2$ is a constant multiplier. $\frac{d}{dx}(x^3)=3x^2$, carried along by $y^2$: $\frac{\partial f}{\partial x}=3x^2{y}^2$ $\partial f/\partial y$: freeze $x$, so $x^3$ is a constant multiplier. $\frac{d}{dy}(y^2)=2y$: $\frac{\partial f}{\partial y}=2x^{3}y$

$\partial f/\partial x$: $y$ is a frozen multiplier on $\sin x$, and $\frac{d}{dx}\sin x=\cos x$; the lone $x$ gives $1$: $\frac{\partial f}{\partial x}=y\cos (x)+1$ $\partial f/\partial y$: freeze $x$, so $\sin x$ is a constant multiplier on $y$, giving $\sin x$; the term $x$ has no $y$ so vanishes: $\frac{\partial f}{\partial y}=\sin (x)$

Let the inside be $u=x^{2}y$. Outer: $\frac{d}{du}{e}^u=e^u$. Then multiply by the inner partial. $\partial u/\partial x=2xy$ (freeze $y$), so $\frac{\partial f}{\partial x}=2xy{e}^{x^{2}y}$ $\partial u/\partial y=x^2$ (freeze $x$), so $\frac{\partial f}{\partial y}=x^2{e}^{x^{2}y}$ The exponential never disappears — it's always $\times e^u$; only the inner factor changes with which variable you moved.

$\partial f/\partial x=2x$ and $\partial f/\partial y=2y$. At $(2,-1)$: ${\frac{\partial f}{\partial x}|}_{(2,-1)}=2(2)=4,{\frac{\partial f}{\partial y}|}_{(2,-1)}=2(-1)=-2$ Reading the signs (blue and pink arrows in the figure above): $+4$ means stepping east uphill (surface rises), and it is steep. $-2$ means stepping north goes downhill, less steeply. So from $(2,-1)$ the fastest way down the bowl is to move west and slightly north — exactly the dashed direction $-\nabla f=(-4,+2)$.

$\partial f/\partial x=y$ and $\partial f/\partial y=x$.

• At $(0,0)$: both are $0$ — the surface is momentarily flat in both axis directions (a saddle passing through here).
• At $(0,3)$: $\partial f/\partial x=3$ (uphill east), $\partial f/\partial y=0$ (flat north). Why the difference: the $x$-slope of $xy$ is the height of $y$ — the more north you already are, the steeper the east–west tilt. This is the hallmark of a coupled function: one variable's slope depends on the other's value.

$\frac{\partial f}{\partial x}=0,\frac{\partial f}{\partial y}=0\text{everywhere.}$ Nothing depends on any input, so nudging any variable changes nothing. The gradient is the zero vector $\nabla f=(0,0)$ — a critical point at every location. This is the limiting "totally flat" case that anchors your intuition: no slope, no direction to descend.

Differentiate once per variable, freezing the other two each time:

• $\partial f/\partial x=2xy$ (only $x^{2}y$ has an $x$)
• $\partial f/\partial y=x^2+z^2$ ($x^{2}y\to x^2$; $y{z}^2\to z^2$; $4z$ has no $y$)
• $\partial f/\partial z=2yz+4$ ($y{z}^2\to 2yz$; $4z\to 4$) $\nabla f=(2xy,x^2+z^2,2yz+4)$ Stacking these three slopes into a vector gives the direction of steepest ascent — recall the gradient definition $\nabla f=\left(\frac{\partial f}{\partial x_1},\dots ,\frac{\partial f}{\partial x_n}\right)$ from the parent note. See 1.2.04-Gradient-and-directional-derivatives.

Let the residual $r=wx+b-t$. Chain rule: $\partial L/\partial \bullet =2r\cdot (\partial r/\partial \bullet )$, with $\partial r/\partial w=x$ and $\partial r/\partial b=1$. $\frac{\partial L}{\partial w}=2(wx+b-t)x,\frac{\partial L}{\partial b}=2(wx+b-t)$ Evaluate: residual $r=2\cdot 3+1-10=-3$. $\frac{\partial L}{\partial w}=2(-3)(3)=-18,\frac{\partial L}{\partial b}=2(-3)=-6$ Both negative $\Rightarrow$ raising $w$ or $b$ would lower the loss right now (residual is $-3$, prediction too small). This is exactly the update signal Linear-regression uses.

Gradient: $\nabla f=(2x,2y)$. At ${\mathbf{p}}_0=(3,4)$: $\nabla f=(6,8)$. Step (subtract $\eta \times$ gradient because we descend — 1.3.01-Gradient-descent): $x_1=3-0.1(6)=2.4,y_1=4-0.1(8)=3.2$ so ${\mathbf{p}}_1=(2.4,3.2)$. Confirm loss dropped: $f(3,4)=9+16=25,f(2.4,3.2)=5.76+10.24=16$ $16<25$ ✅. The point slid toward the bowl's bottom $(0,0)$ — this is the first blue arrow in the figure above.

Here we nudge $y$, so $y\to y+h$ while $x$ stays put: $\frac{\partial f}{\partial y}={\lim }_{h\to 0}\frac{x^2(y+h)-x^{2}y}{h}$ Why this step: the definition perturbs only $y$; $x^2$ is untouched. $={\lim }_{h\to 0}\frac{x^{2}y+x^{2}h-x^{2}y}{h}={\lim }_{h\to 0}\frac{x^{2}h}{h}={\lim }_{h\to 0}{x}^2=x^2$ Why this step: the $x^{2}y$ terms cancel, $h$ factors out and cancels, then $h\to 0$ leaves $x^2$. Shortcut check: treat $x^2$ as a constant multiplier on $y$; $\frac{d}{dy}(x^{2}y)=x^2$. ✅ Match.

Residual $r=w_1{x}_1+w_2{x}_2+b-t=1(1)+1(2)+0-5=-2$. Each partial is $2r$ times the inner factor ($\partial r/\partial w_1=x_1$, $\partial r/\partial w_2=x_2$, $\partial r/\partial b=1$): $\frac{\partial L}{\partial w_1}=2(-2)(1)=-4,\frac{\partial L}{\partial w_2}=2(-2)(2)=-8,\frac{\partial L}{\partial b}=2(-2)=-4$ $\nabla L=(-4,-8,-4)$ Feature $x_2$ is twice as large, so its weight's gradient is twice as strong — the model's slope in that direction is amplified by its input. This is the multivariable pattern that scales up to real networks via 1.2.05-Chain-rule-multivariable.