Gradients and directional derivatives: Question bank

Before you start, recall the three characters:

A partial derivative $\partial f / \partial x_{i}$ — a slope along one axis (see Partial derivatives).
The gradient $\nabla f$ — a vector stacking all partials, pointing steepest-uphill.
The directional derivative $D_{u} f = \nabla f \cdot u$ — a scalar slope in a chosen unit direction $u$ , built through the Chain rule and read as a dot product.

Spot the error

D_{u} f = \nabla f \cdot u = ∥\nabla f ∥∥ u ∥ cos θ

, so it's biggest when

u

is big."

The error is treating

∥ u ∥

as free. We fix

∥ u ∥ = 1

; the only free quantity is

θ

, so the size of

D_{u} f

is controlled by alignment, not length.

"The gradient equals the slope of the hill, so it's just one number."

Confuses the vector

\nabla f

with the scalar slope. The slope in a chosen direction is the number

\nabla f \cdot u

; the gradient itself is the vector encoding all such slopes.

"To get the directional derivative toward

(4, 6)

from

(1, 2)

, dot

\nabla f

with

(4, 6)

You must use the direction of travel,

(4, 6) - (1, 2) = (3, 4)

, and then normalize to

(3/5, 4/5)

before dotting. Using the raw endpoint gives a meaningless number.

g (t) = f (a + t u)

, and

g^{'} (t) = f^{'} (a + t u)

f

has many inputs, so there is no single

f^{'}

. The Chain rule gives

g^{'} (t) = \sum_{i} \frac{\partial f}{\partial x _{i}} u_{i} = \nabla f \cdot u

— a sum over all coordinates, not one derivative.

\nabla f

tells you where the maximum of

f

is."

It only tells you the local uphill direction, not the location of any maximum. At an actual maximum

\nabla f = 0

; away from it the gradient just points the way that increases

f

fastest right here.

"Gradient descent uses

- \nabla f

, so bigger gradient always means we take a bigger step and that's good."

The step is

- η \nabla f

; a huge gradient with a large

η

can overshoot the minimum and diverge. Magnitude signals steepness, not a guarantee of progress — step size

η

must be controlled.

"Since the gradient is perpendicular to contours, the directional derivative along the gradient is zero."

Reversed. It is zero along the contour (perpendicular to

\nabla f

) and maximal along the gradient (

cos θ = 1

Why questions

Why must

u

be a unit vector in

D_{u} f = \nabla f \cdot u

So the answer is a slope per unit distance travelled; a longer

u

would scale the rate by its length, conflating distance with steepness.

Why do we use a dot product rather than some other combination of

\nabla f

and

u

Because the Chain rule on

g (t) = f (a + t u)

produces exactly

\sum_{i} \frac{\partial f}{\partial x _{i}} u_{i}

, which is the dot product — it falls out of the maths, we don't impose it.

Why does gradient descent move in

- \nabla f

instead of

+ \nabla f

+ \nabla f

is steepest ascent (increases loss); training wants to reduce loss, so it walks the steepest-descent direction

- \nabla f

Why is the maximum directional derivative exactly

∥\nabla f ∥

From

∥\nabla f ∥ cos θ

, the largest value occurs at

cos θ = 1

(align with

\nabla f

), leaving just

∥\nabla f ∥

Why is the gradient perpendicular to level sets?

Along a level set

f

doesn't change, so its directional derivative is

0

, i.e.

\nabla f \cdot u = 0

— the only way a nonzero

\nabla f

dots to zero is by being orthogonal to

u

Why do we treat all other variables as constant when taking a partial derivative?

A partial measures change along one coordinate axis; freezing the others isolates that axis so the north-slope can't contaminate the east-slope.

Why does the sign of

D_{u} f

tell you if you're going up or down?

Its sign is the sign of

cos θ

: positive means

u

leans uphill, negative means downhill, zero means level.

Edge cases

At a point where

\nabla f = 0

, what is

D_{u} f

for every direction?

Zero in all directions — the surface is momentarily flat (a critical point: min, max, or saddle), so no direction has first-order slope.

\nabla f = 0

, does that guarantee a minimum?

No. It only marks a critical point; it could be a maximum or a saddle. Distinguishing them needs curvature — the Hessian.

For

f (x, y) = x^{2} + y^{2}

at the origin

(0, 0)

, what direction is "steepest ascent"?

There is none —

\nabla f = (0, 0)

at the exact bottom of the bowl, so every direction has slope

0

and the notion of "steepest" is undefined.

Along a perfectly flat plateau (constant

f

), what does the gradient look like everywhere?

\nabla f = 0

throughout, since no direction changes

f

; there are no contours to be perpendicular to because every point sits at the same level.

f

is linear, e.g.

f (x, y) = 3 x + 4 y

, how does

\nabla f

vary from point to point?

It doesn't —

\nabla f = (3, 4)

everywhere; a plane has the same steepest-uphill direction at every point, and the contours are parallel straight lines.

What happens to

D_{u} f

if you reverse the direction to

- u

It flips sign:

\nabla f \cdot (- u) = - (\nabla f \cdot u)

. Walking backwards up a slope is descending at the same rate.

Can the gradient exist while some directional derivatives behave badly (e.g. at a sharp corner)?

Yes — at kinks like the tip of

f = x^{2} + y^{2}

, partials may fail to exist or disagree by direction, so the neat formula

D_{u} f = \nabla f \cdot u

only holds where

f

is differentiable (smooth).

Recall One-line self-test

Give the fastest-descent direction and its rate for $f$ at a point where $\nabla f = (8, 7)$ . Direction $:::$ $- (8, 7) / 113$ ; rate of change $= - 113 \approx - 10.63$ (steepest possible decrease).

Connections

Partial derivatives — the per-axis slopes these traps keep confusing with the gradient.
Chain rule — the reason $D_{u} f$ is a dot product.
Dot product and projections — the geometry of $∥\nabla f ∥ cos θ$ .
Level sets and contours — perpendicularity trap lives here.
Gradient descent — why $- \nabla f$ , and the step-size edge case.
Jacobian and Hessian — needed when $\nabla f = 0$ to classify the point.

Question bank — Gradients and directional derivatives

True or false — justify