Derivatives and rules (product, quotient, chain): Exercises

Reminders you will lean on (all built in the parent note):

Two derivatives we will reuse (each is itself a chain-rule fact you can re-derive): $\frac{d}{d x} sin x = cos x, \frac{d}{d x} cos x = - sin x, \frac{d}{d x} e^{x} = e^{x} .$

Level 1 — Recognition

Goal: name the rule and apply it once, no nesting deeper than one layer.

Problem 1.1

Differentiate $f (x) = x^{3} cos x$ .

Recall Solution 1.1

What is this? A product of two things I can each differentiate. So: product rule. Let $u = x^{3} \Rightarrow u^{'} = 3 x^{2}$ , and $v = cos x \Rightarrow v^{'} = - sin x$ . $f^{'} (x) = u^{'} v + u v^{'} = 3 x^{2} cos x + x^{3} (- sin x) = 3 x^{2} cos x - x^{3} sin x$ Why the minus? It is not the product rule flipping — it comes from $v^{'} = - sin x$ . Keep the two effects separate.

Problem 1.2

Differentiate $g (x) = \frac{x ^{2}}{x + 1}$ .

Recall Solution 1.2

What is this? A quotient. Use "Lo D-Hi minus Hi D-Lo over Lo-Lo." Top $u = x^{2} \Rightarrow u^{'} = 2 x$ ; bottom $v = x + 1 \Rightarrow v^{'} = 1$ . $g^{'} (x) = \frac{u ^{'} v - u v ^{'}}{v ^{2}} = \frac{2 x ( x + 1 ) - x ^{2} ( 1 )}{( x + 1 ) ^{2}} = \frac{2 x ^{2} + 2 x - x ^{2}}{( x + 1 ) ^{2}} = \frac{x ^{2} + 2 x}{( x + 1 ) ^{2}}$

Problem 1.3

Differentiate $h (x) = (5 x - 2)^{4}$ .

Recall Solution 1.3

What is this? A shell $()^{4}$ wrapped around an inside $5 x - 2$ . That is a chain. Outer $f (u) = u^{4} \Rightarrow f^{'} (u) = 4 u^{3}$ ; inner $u = 5 x - 2 \Rightarrow u^{'} = 5$ . $h^{'} (x) = 4 u^{3} \cdot 5 = 4 (5 x - 2)^{3} \cdot 5 = 20 (5 x - 2)^{3}$

Level 2 — Application

Goal: turn the crank cleanly, including one nested chain.

Problem 2.1

Differentiate $f (x) = x^{2} e^{3 x}$ .

Recall Solution 2.1

Product, with the second factor needing a small chain. $u = x^{2} \Rightarrow u^{'} = 2 x$ . For $v = e^{3 x}$ : outer $e^{□}$ (derivative $e^{□}$ ), inner $3 x$ (derivative $3$ ), so $v^{'} = e^{3 x} \cdot 3 = 3 e^{3 x}$ . $f^{'} (x) = 2 x e^{3 x} + x^{2} \cdot 3 e^{3 x} = e^{3 x} (2 x + 3 x^{2})$ Why factor out $e^{3 x}$ ? It never vanishes, so pulling it out shows the zeros cleanly ( $x = 0$ and $x = - 2/3$ ).

Problem 2.2

Differentiate $g (x) = sin (x^{2})$ .

Recall Solution 2.2

Chain. Outer $sin (□)$ has derivative $cos (□)$ ; inner $x^{2}$ has derivative $2 x$ . $g^{'} (x) = cos (x^{2}) \cdot 2 x = 2 x cos (x^{2})$ Why not $cos (x^{2})$ alone? That would pretend the inside is frozen. The $2 x$ is how fast the inside moves — the gear ratio.

Problem 2.3

Differentiate $h (x) = \frac{e ^{x}}{x ^{2} + 1}$ and evaluate $h^{'} (0)$ .

Recall Solution 2.3

Quotient. $u = e^{x} \Rightarrow u^{'} = e^{x}$ ; $v = x^{2} + 1 \Rightarrow v^{'} = 2 x$ . $h^{'} (x) = \frac{e ^{x} ( x ^{2} + 1 ) - e ^{x} ( 2 x )}{( x ^{2} + 1 ) ^{2}} = \frac{e ^{x} ( x ^{2} - 2 x + 1 )}{( x ^{2} + 1 ) ^{2}} = \frac{e ^{x} ( x - 1 ) ^{2}}{( x ^{2} + 1 ) ^{2}}$ At $x = 0$ : $\frac{e ^{0} ( 0 - 1 ) ^{2}}{( 0 + 1 ) ^{2}} = \frac{1 \cdot 1}{1} = 1$ . Nice bonus: the numerator is $e^{x} (x - 1)^{2} \geq 0$ , so $h$ is never decreasing — flat only at $x = 1$ .

Level 3 — Analysis

Goal: read meaning — slopes, maxima, monotonicity — off the derivative.

Problem 3.1

For $f (x) = \frac{x}{x ^{2} + 4}$ , find all $x$ where the tangent line is horizontal, and classify each as a peak or valley.

Figure — Derivatives and rules (product, quotient, chain)

Recall Solution 3.1

Horizontal tangent means ==slope $= 0$ ==, i.e. $f^{'} (x) = 0$ . Quotient rule: $u = x, u^{'} = 1$ ; $v = x^{2} + 4, v^{'} = 2 x$ . $f^{'} (x) = \frac{1 ( x ^{2} + 4 ) - x ( 2 x )}{( x ^{2} + 4 ) ^{2}} = \frac{4 - x ^{2}}{( x ^{2} + 4 ) ^{2}}$ The denominator is always positive, so $f^{'} = 0$ exactly when the numerator $4 - x^{2} = 0 \Rightarrow x = \pm 2$ . Classify by sign of $f^{'}$ (look at the figure): the numerator $4 - x^{2}$ is a downward parabola — positive between $- 2$ and $2$ , negative outside.

At $x = - 2$ : slope goes $- \to +$ (falling then rising) → valley (minimum). Value $f (- 2) = \frac{- 2}{8} = - \frac{1}{4}$ .
At $x = + 2$ : slope goes $+ \to -$ (rising then falling) → peak (maximum). Value $f (2) = \frac{2}{8} = \frac{1}{4}$ .

Problem 3.2

Show that $σ (x) = \frac{1}{1 + e ^{- x}}$ is strictly increasing everywhere, and find where its slope is largest.

Recall Solution 3.2

From the parent note, $σ^{'} (x) = σ (x) (1 - σ (x))$ . Since $0 < σ (x) < 1$ for every real $x$ (it is a probability-like squash), both factors are positive ⇒ $σ^{'} (x) > 0$ everywhere ⇒ strictly increasing. Where is the slope largest? $σ^{'} = σ (1 - σ)$ is a downward parabola in the variable $σ$ , peaking at $σ = \frac{1}{2}$ . And $σ (x) = \frac{1}{2}$ when $x = 0$ . Maximum slope $= \frac{1}{2} \cdot \frac{1}{2} = \frac{1}{4}$ at $x = 0$ . Why ML cares: this $\frac{1}{4}$ cap is the seed of the vanishing gradient problem — see Activation Functions and Backpropagation.

Level 4 — Synthesis

Goal: all three rules braided into one expression.

Problem 4.1

Differentiate $f (x) = x^{2} sin (2 x)$ .

Recall Solution 4.1

Product on the outside ( $u = x^{2}$ , $v = sin 2 x$ ); the $v$ needs a chain. $u^{'} = 2 x$ . $v^{'} = cos (2 x) \cdot 2 = 2 cos (2 x)$ . $f^{'} (x) = 2 x sin (2 x) + x^{2} \cdot 2 cos (2 x) = 2 x sin (2 x) + 2 x^{2} cos (2 x)$

Problem 4.2

Differentiate $g (x) = \frac{( x ^{2} + 1 ) ^{3}}{x}$ .

Recall Solution 4.2

Quotient outside; the top needs a chain. $u = (x^{2} + 1)^{3} \Rightarrow u^{'} = 3 (x^{2} + 1)^{2} \cdot 2 x = 6 x (x^{2} + 1)^{2}$ . $v = x \Rightarrow v^{'} = 1$ . $g^{'} (x) = \frac{6 x ( x ^{2} + 1 ) ^{2} \cdot x - ( x ^{2} + 1 ) ^{3} \cdot 1}{x ^{2}} = \frac{( x ^{2} + 1 ) ^{2} [ 6 x ^{2} - ( x ^{2} + 1 ) ]}{x ^{2}}$ Simplify the bracket: $6 x^{2} - x^{2} - 1 = 5 x^{2} - 1$ . $g^{'} (x) = \frac{( x ^{2} + 1 ) ^{2} ( 5 x ^{2} - 1 )}{x ^{2}}$ Why factor before simplifying? Pulling out $(x^{2} + 1)^{2}$ turns a scary difference into a tidy product — and reveals the zeros at $x = \pm \frac{1}{5}$ .

Problem 4.3

Differentiate $h (x) = (x e^{x})^{2}$ two ways and check they agree.

Recall Solution 4.3

Way A — chain on the outer square. Let $u = x e^{x}$ . Then $h = u^{2} \Rightarrow h^{'} = 2 u \cdot u^{'}$ . Inner product: $u^{'} = (x e^{x})^{'} = 1 \cdot e^{x} + x \cdot e^{x} = e^{x} (1 + x)$ . $h^{'} = 2 (x e^{x}) \cdot e^{x} (1 + x) = 2 x e^{2 x} (1 + x)$ Way B — expand first. $h = x^{2} e^{2 x}$ . Product rule: $(x^{2})^{'} = 2 x$ , $(e^{2 x})^{'} = 2 e^{2 x}$ . $h^{'} = 2 x e^{2 x} + x^{2} \cdot 2 e^{2 x} = 2 e^{2 x} (x + x^{2}) = 2 x e^{2 x} (1 + x)$ Same answer $\Rightarrow h^{'} (x) = 2 x e^{2 x} (1 + x)$ . ✓

Level 5 — Mastery

Goal: use derivatives the way ML actually uses them — composed layers, gradients, and optimization.

Problem 5.1 (Backprop by hand)

A tiny network computes $L (w) = (σ (w x) - t)^{2}$ for fixed input $x = 1$ and target $t = 0$ , where $σ$ is the sigmoid. Find $\frac{d L}{d w}$ at $w = 0$ .

Recall Solution 5.1

This is a 3-gear chain: $w \to z = w x \to a = σ (z) \to L = (a - t)^{2}$ . Multiply the gear ratios. $\frac{d L}{d w} = 2 (a - t) \frac{d L}{d a} \cdot σ (z) (1 - σ (z)) \frac{d a}{d z} \cdot x \frac{d z}{d w}$ Evaluate at $w = 0$ , $x = 1$ , $t = 0$ :

$z = w x = 0$ , so $a = σ (0) = \frac{1}{2}$ .
$\frac{d L}{d a} = 2 (a - t) = 2 (\frac{1}{2} - 0) = 1$ .
$\frac{d a}{d z} = σ (0) (1 - σ (0)) = \frac{1}{2} \cdot \frac{1}{2} = \frac{1}{4}$ .
$\frac{d z}{d w} = x = 1$ . $\frac{d L}{d w} = 1 \cdot \frac{1}{4} \cdot 1 = \frac{1}{4}$ Meaning for training: the gradient is positive, so Gradient Descent would decrease $w$ to shrink the loss. Each factor is one layer's Backpropagation message. See also Gradient and Partial Derivatives.

Problem 5.2 (Product-of-many pattern)

Let $P (x) = \prod$ of three factors: $P (x) = (x + 1) (x + 2) (x + 3)$ . Show that $\frac{P ^{'} ( x )}{P ( x )} = \frac{1}{x + 1} + \frac{1}{x + 2} + \frac{1}{x + 3}$ (the "log-derivative" trick), then compute $P^{'} (0)$ .

Recall Solution 5.2

Why this identity? Repeatedly applying the product rule to a product of $n$ factors gives a sum where each term differentiates one factor and leaves the rest. Dividing by $P$ turns each term into $\frac{( factor ) ^{'}}{factor}$ . Concretely, with $u = x + 1, v = x + 2, w = x + 3$ (each derivative $= 1$ ): $P^{'} = u^{'} v w + u v^{'} w + uv w^{'} = v w + u w + uv .$ Divide by $P = uv w$ : $\frac{P ^{'}}{P} = \frac{1}{u} + \frac{1}{v} + \frac{1}{w} = \frac{1}{x + 1} + \frac{1}{x + 2} + \frac{1}{x + 3}$ . ✓ Now $P (0) = 1 \cdot 2 \cdot 3 = 6$ and $\frac{P ^{'} ( 0 )}{P ( 0 )} = \frac{1}{1} + \frac{1}{2} + \frac{1}{3} = \frac{11}{6}$ . $P^{'} (0) = 6 \cdot \frac{11}{6} = 11$ Sanity by expansion: $P = x^{3} + 6 x^{2} + 11 x + 6 \Rightarrow P^{'} = 3 x^{2} + 12 x + 11 \Rightarrow P^{'} (0) = 11$ . ✓

Problem 5.3 (One optimization step)

The loss $L (w) = (w - 3)^{2} + 2$ is minimized by gradient descent with learning rate $η = 0.1$ , starting at $w_{0} = 0$ . Compute $w_{1}$ (one update). Where is the true minimum, and is the step moving toward it?

Recall Solution 5.3

Gradient: $L^{'} (w) = 2 (w - 3)$ . At $w_{0} = 0$ : $L^{'} (0) = 2 (0 - 3) = - 6$ . Update rule (from Gradient Descent): step against the gradient. $w_{1} = w_{0} - η L^{'} (w_{0}) = 0 - 0.1 \cdot (- 6) = 0.6$ True minimum: $L^{'} (w) = 0 \Rightarrow w = 3$ . Since we moved from $0$ to $0.6$ , we stepped toward $3$ . ✓ Why negative gradient? The derivative points uphill (direction of steepest increase); subtracting it walks downhill toward smaller loss.

6. Score yourself

Recall Rubric

Got L1–L2 (5 problems): you can apply each rule. Solid foundation.
Add L3 (2 problems): you can interpret derivatives — you're ready for optimization.
Add L4 (3 problems): you can handle any algebraic mess.
Add L5 (3 problems): you can do backprop and a descent step by hand — genuine ML mastery.

Recall Quick self-quiz

Slope of $x^{3} cos x$ ::: $3 x^{2} cos x - x^{3} sin x$ Where is $\frac{x}{x ^{2} + 4}$ maximal ::: at $x = 2$ , value $\frac{1}{4}$ Max slope of sigmoid and where ::: $\frac{1}{4}$ , at $x = 0$ One GD step for $L = (w - 3)^{2} + 2$ , $w_{0} = 0$ , $η = 0.1$ ::: $w_{1} = 0.6$ $P^{'} (0)$ for $(x + 1) (x + 2) (x + 3)$ ::: $11$

Connections

Parent: the three rules
Power Rule and Basic Derivatives — the building block
Limits and Continuity — what a derivative rests on
Gradient and Partial Derivatives · Backpropagation · Gradient Descent · Activation Functions