Gradient as direction of steepest ascent: Question bank

These items probe the ideas in the parent note: the meaning of $\nabla f$ (the gradient, the vector of partial derivatives), the directional derivative $D_{u} f = \nabla f \cdot u$ (valid only for a unit vector $u$ , meaning $∥ u ∥ = 1$ ), its link to the dot product, and how $\nabla f$ sits against level curves.

True or false — justify

The gradient always points toward the maximum (highest peak) of the function.

False. It points in the direction of locally fastest increase at that one point; it knows nothing about far-away peaks. It's a local slope arrow, not a GPS to the summit.

\nabla f = 0

at a point, then

f

has a maximum there.

False. A zero gradient means a critical point — could be a max, a min, or a saddle. All you know is that every directional derivative is

0

there, so the surface is momentarily flat.

For a unit

u

, the maximum value of

D_{u} f = \nabla f \cdot u

over all such

u

equals

∥\nabla f ∥

True. Since

D_{u} f = ∥\nabla f ∥ cos θ

(using

∥ u ∥ = 1

) and

cos θ \leq 1

, the largest value is

∥\nabla f ∥

, hit when

u

aligns with

\nabla f

(

θ = 0

The gradient is tangent to the level curve through a point.

False. It is perpendicular to the level curve. Along the curve

f

doesn't change, so

D_{u} f = 0 \Rightarrow cos θ = 0 \Rightarrow θ = 9 0^{\circ}

If two directions give the same directional derivative, they must be the same direction.

False.

cos θ

is symmetric, so two unit vectors making equal angles on either side of

\nabla f

give the same

D_{u} f

. They're mirror images across the gradient line.

Steepest descent is a different, independent direction that you must compute separately.

False. It is simply

- \nabla f

, the exact opposite arrow (

θ = π

cos θ = - 1

). One computation gives you both.

Doubling every input weight scaling of

f

(replacing

f

2 f

) changes the direction of steepest ascent.

False.

\nabla (2 f) = 2\nabla f

— same direction, twice the length. Direction of steepest ascent is unchanged; only the rate

∥\nabla f ∥

doubles.

D_{u} f

(for a unit

u

) can be negative.

True. If you step "downhill",

cos θ < 0

D_{u} f < 0

; the most negative it gets is

- ∥\nabla f ∥

(steepest descent).

Spot the error

D_{v} f = \nabla f \cdot v

where

v = (3, 4)

points the way I want."

Error:

v

isn't a unit vector (

∥ v ∥ = 5

). The formula

D_{u} f = \nabla f \cdot u

needs

∥ u ∥ = 1

, so normalise first:

u = (3, 4) /5

. Otherwise the rate is inflated by a factor of

5

"Since

\nabla f

points uphill, walking along

\nabla f

keeps me on the same contour line."

Error: contour lines are level sets where

f

is constant; walking along

\nabla f

maximally changes

f

. The gradient is perpendicular to contours, not along them.

"The directional derivative depends only on the direction, not the point."

Error:

\nabla f

is evaluated at the point

x = (x, y)

, so

D_{u} f = \nabla f (x) \cdot u

changes as you move

x

. Both the point and the direction matter.

∥\nabla f ∥

is the slope in the

x

-direction."

Error: the slope in the

x

-direction is the partial

f_{x}

. The magnitude

∥\nabla f ∥ = f_{x}^{2} + f_{y}^{2}

is the maximum slope over all directions, which is generally larger than

∣ f_{x} ∣

"I found

D_{u} f = 5

(unit

u

) but

∥\nabla f ∥ = 4

— great."

Error: impossible. No directional derivative from a unit direction can exceed

∥\nabla f ∥

, since

D_{u} f = ∥\nabla f ∥ cos θ \leq ∥\nabla f ∥

. A recomputation is needed.

"The gradient of

f (x, y, z)

is a scalar giving the overall steepness."

Error: the gradient is a vector

(f_{x}, f_{y}, f_{z})

. Its magnitude is the scalar steepness; the vector also carries the direction.

"To get steepest descent I take

1/\nabla f

Error: you can't divide by a vector. Steepest descent is

- \nabla f

(negate it), and the unit descent direction is

- \nabla f /∥\nabla f ∥

Why questions

Why must

u

be a unit vector in

D_{u} f = \nabla f \cdot u

So the rate is measured per unit of distance travelled. If

∥ u ∥ \neq = 1

, the dot product also scales by

∥ u ∥

, mixing "how far" into "how fast".

Why is the steepest ascent direction

\nabla f /∥\nabla f ∥

rather than

\nabla f

itself?

The direction is the unit-length version; dividing by

∥\nabla f ∥

strips off the magnitude so you keep only "which way".

\nabla f

points the same way but isn't unit length.

Why does perpendicularity to level curves follow from the dot-product formula?

Along a level curve

f

is constant, so its rate of change

D_{u} f = \nabla f \cdot u = 0

. A zero dot product (with nonzero vectors) means a

9 0^{\circ}

angle.

Why do we use the chain rule to derive

D_{u} f

instead of the limit each time?

The limit would have to be recomputed for every direction. The chain rule expresses

D_{u} f

once, in terms of partials we already have, valid for all directions.

Why does the gradient descent algorithm step in

- \nabla f

Because

- \nabla f

is the direction of fastest decrease, letting the algorithm reduce a loss as quickly as possible per step — see Gradient Descent (Machine Learning).

Why is the maximum rate the magnitude of the gradient, a purely geometric fact?

Because

\nabla f \cdot u = ∥\nabla f ∥ cos θ

and the biggest

cos θ

can be is

1

; the direction detail vanishes, leaving just the length

∥\nabla f ∥

Edge cases

At a critical point where

\nabla f = 0

, what is the direction of steepest ascent?

Undefined. Every direction gives

D_{u} f = 0

, and

\nabla f /∥\nabla f ∥ = 0 /0

is meaningless — the surface is flat to first order in all directions.

For a linear field

f (x, y) = a x + b y

, how does

\nabla f

change from point to point?

It doesn't:

\nabla f = (a, b)

everywhere. The steepness and steepest direction are constant across the whole plane (a tilted flat sheet).

∥\nabla f ∥

is tiny but nonzero, what does that say about the terrain?

The slope is nearly flat — even the steepest uphill direction rises very slowly. The gradient direction is still well-defined, just short.

Can two perpendicular directions both give positive directional derivatives?

Yes, as long as neither is along a level curve. If

\nabla f

lies strictly between them, both make an angle under

9 0^{\circ}

with

\nabla f

, so both

cos θ > 0

At a point on a level curve, is

D_{u} f

zero for the tangent direction only, or for a whole range?

Only for the two tangent directions (

\pm

along the curve). Every other direction has

cos θ \neq = 0

, giving a nonzero rate — positive on the uphill side, negative on the downhill side.

What happens to steepest ascent on a perfectly flat region (

f = const

everywhere)?

\nabla f = 0

throughout, so there is no direction of increase — every step keeps

f

the same. "Steepest ascent" is vacuous there.

u

makes a

6 0^{\circ}

angle with

\nabla f

, what fraction of the maximum rate do you get?

cos 6 0^{\circ} = \frac{1}{2}

, so exactly half of

∥\nabla f ∥

. Angle alone determines the fraction, independent of how large the gradient is.

Recall One-line self-test

The gradient's direction answers "which way up?"; its length answers "how steep?"; its dot with a unit direction answers "how fast along this way?".

Connections

Gradient as direction of steepest ascent
Directional Derivative
Partial Derivatives
Multivariable Chain Rule
Dot Product and Angle
Level Curves and Contour Maps
Gradient Descent (Machine Learning)
Tangent Plane and Linearization