Foundations — Gradient as direction of steepest ascent

Before you can trust the sentence "$\nabla f$ points in the direction of steepest ascent", you need to already own every symbol in it. This page tears the topic down to its atoms and rebuilds them one at a time. If a symbol appears here, we define it in plain words, draw its picture, and say why the topic can't live without it.

0. The picture we keep returning to: a hill

Every idea below lives on top of the same mental image, so let's draw it first.

Look at the surface. The floor is the flat $(x,y)$ plane — a map you could hold in your hands. Above every floor point $(x,y)$ the surface rises to some height. That height is what the letter $f$ will name. Keep this in your head: floor = inputs, height = output.

1. The symbols $x$, $y$ — a place on the floor

Picture. A dot on the floor of the hill figure. Two dots = two different places you could be standing.

Why the topic needs it. The whole question is "at this place, which way is uphill?" You cannot ask that without a way to name the place. $(x,y)$ is that name.

2. The symbol $f$ and the phrase "scalar field" — the height at a place

Picture. Stand at $(x,y)$, look straight up to the surface, read off the height. That number is $f(x,y)$.

Why the topic needs it. "Increases fastest" is meaningless unless something is increasing. That something is the height $f$. The parent's opening line — "the function increases fastest" — is talking about $f$.

3. Level curves — drawing the hill flat

You can't carry a 3-D hill around, so mapmakers flatten it. Slice the hill at height $=3$, at height $=6$, at height $=9$, and mark where each slice meets the floor. Each mark is a loop of "all places at exactly this height".

Picture. The nested loops on the right of the figure — like the rings on a topographic map, or the edge of the water as a flooding hill fills up.

Why the topic needs it. The parent claims $\nabla f$ is perpendicular to these loops. To even parse that sentence you must know what the loops are. They are also the direction where $f$ does not change — the flat sideways walk.

See Level Curves and Contour Maps for more.

4. Vectors and $\mathbf{u}$ — an arrow, a direction to step

Picture. An arrow lying flat on the floor, pointing the way you're about to walk.

To make any arrow into a unit arrow ("normalise"): divide it by its own length. $\mathbf{u}=\frac{\mathbf{v}}{\Vert \mathbf{v}\Vert }$

Why a unit vector? The directional derivative measures rate "per unit of distance travelled". If your step arrow were $2$ units long, you'd cover twice the ground per step and the rate would come out doubled — a lie about the steepness. Shrinking to length $1$ keeps the measurement honest. This is exactly the parent's "normalise before you dot".

5. The dot product $\mathbf{a}\cdot \mathbf{b}$ — how much two arrows agree

Picture. Two arrows sharing a tail with an angle $\theta$ between them. The dot product is large and positive when they nearly point the same way ($\theta$ small), zero when they're at a right angle, and negative when they oppose.

Why the topic needs it. The parent's boxed result is $D_{\mathbf{u}}f=\nabla f\cdot \mathbf{u}$. The dot product is the bridge from "gradient arrow + step arrow" to "a single rate number". And the geometric form is what makes $\cos \theta =1$ pop out as the winner.

More at Dot Product and Angle.

6. Limits and the difference quotient — rate of change from scratch

Picture. Two nearby points on a curve joined by a straight line (a secant). As the second point slides toward the first, that line tips over into the tangent — the true slope right there.

Why the topic needs it. The parent defines the directional derivative with a limit: $D_{\mathbf{u}}f={\lim }_{h\to 0}\frac{f(\mathbf{x}+h\mathbf{u})-f(\mathbf{x})}{h}$ Without "limit" and "difference quotient" this line is unreadable. It says: nudge a tiny amount $h$ in direction $\mathbf{u}$, see how the height changed per unit of nudge.

7. Partial derivatives $f_x,f_y$ — slope if you only move east, or only north

Picture. Slice the hill with a vertical wall running east–west; the edge of that slice is an ordinary 1-D curve, and $f_x$ is its slope. Turn the wall $90^{\circ }$ for $f_y$.

Why the topic needs it. These two slopes are the ingredients of the gradient. If you know how steep it is going pure-east and pure-north, you can work out how steep it is in any direction — that's the next tool.

See Partial Derivatives.

8. The multivariable chain rule — stitching the partials along a path

Picture. Walking diagonally is "a bit of east + a bit of north". The chain rule says: your uphill rate = (east-slope × how fast you go east) + (north-slope × how fast you go north).

Why the topic needs it. This is the trick that turns the scary limit-definition of $D_{\mathbf{u}}f$ into the friendly $f_x{u}_1+f_y{u}_2$, without taking a fresh limit for every direction. It's the single most important step in the whole derivation.

See Multivariable Chain Rule.

9. The gradient $\nabla f$ — the two slopes bundled into one arrow

Picture. An arrow on the floor whose eastward part is the east-slope and whose northward part is the north-slope. Because steeper terrain makes longer components, the arrow is longer where the hill is steeper and it leans toward the steepest side.

Why the topic needs it. This is the topic. Everything on the parent page reads this arrow: its direction (steepest ascent), its length (max rate), and its right angle to the level curves.

How these feed the topic

Equipment checklist

Cover the right side. Can you state each from memory?

Connections

• Gradient as direction of steepest ascent
• Directional Derivative
• Partial Derivatives
• Multivariable Chain Rule
• Dot Product and Angle
• Level Curves and Contour Maps
• Gradient Descent (Machine Learning)
• Tangent Plane and Linearization