Gradient vector ∇f — definition, properties: Visual walkthrough

Step 1 — A function of two variables is a landscape

WHAT. Take a rule $f$ that eats two numbers $(x, y)$ and returns one number, its "height". Call the height $z = f (x, y)$ .

WHY. Before we can talk about "slope" or "steepest", we need something that has a slope. One input gives a curve; two inputs give a surface — a hill. We must picture the hill first, because every later idea is "how does the height change as I walk?"

PICTURE. Below, the input $(x, y)$ lives on the flat grey floor. Directly above each floor point, the height $z$ lifts up into a smooth hill. Follow the yellow dot: it sits at a floor point $p = (p_{1}, p_{2})$ , and the vertical yellow stick measures its height $f (p)$ .

Figure — Gradient vector ∇f — definition, properties

Step 2 — Two special walks: along $x$ , along $y$

WHAT. Freeze $y = p_{2}$ and walk only east (change $x$ ). The height traces a 1-D curve — a single slice of the hill. Its slope is the partial derivative $f_{x}$ . Do the same walking north (freeze $x$ ): its slope is $f_{y}$ .

WHY. A surface has infinitely many directions to walk in. That's too many to grasp at once. So we start with the two simplest walks — straight along each axis — because on those, the problem collapses to ordinary one-variable slope, which we already understand.

PICTURE. Two coloured slices cut through the yellow point. The blue slice runs east–west (only $x$ moves); its steepness at $p$ is $f_{x}$ . The green slice runs north–south (only $y$ moves); its steepness is $f_{y}$ . Each is just a hill-cross-section — an ordinary curve with an ordinary slope.

Step 3 — But we want to walk in ANY direction

WHAT. Pick any direction on the floor and call it $\overset{u}{^} = (u_{1}, u_{2})$ . The little hat means it is a unit vector: its length is exactly $1$ , i.e. $u_{1}^{2} + u_{2}^{2} = 1$ . We ask: how fast does height change if I walk this way?

WHY a unit vector? "Direction" and "how far" are two different questions. We want the pure rate of climb per metre walked. If our arrow were 2 metres long, we'd secretly be measuring two metres of walking and double-counting. Fixing the length to $1$ strips away the "how far" so only the "which way" remains.

PICTURE. From the yellow floor point, a red arrow $\overset{u}{^}$ of length $1$ points in some chosen direction. The blue ( $f_{x}$ ) and green ( $f_{y}$ ) axis-arrows are also length $1$ . Notice $\overset{u}{^}$ sits between them — any direction is a blend of "some east, some north".

Step 4 — Turn the 2-D walk into a 1-D walk

WHAT. Define a single-variable function $g (h) = f (p + h \overset{u}{^})$ — height as a function only of how far, $h$ , we've walked along the red arrow. Then $D_{\overset{u}{^}} f = g^{'} (0)$ : the slope of $g$ at the moment we set off.

WHY. We know one-variable calculus cold. The trick is to reduce the 2-D question to a 1-D one by parametrising the straight red path with one number $h$ . Now "rate of climb along $\overset{u}{^}$ " is literally " $g^{'} (0)$ ", an ordinary derivative.

PICTURE. The red path on the floor is lifted onto the hill (the red curve on the surface). Squash your view along that path and it becomes a plain 2-D graph of $g$ versus $h$ : an ordinary curve whose slope at $h = 0$ is what we want.

Step 5 — The chain rule cracks it open

WHAT. As $h$ grows, the $x$ -coordinate is $p_{1} + h u_{1}$ and the $y$ -coordinate is $p_{2} + h u_{2}$ . The Chain rule (multivariable) says the total rate of height change is: (rate along $x$ ) × (how fast $x$ moves) + (rate along $y$ ) × (how fast $y$ moves).

$g^{'} (0) = height’s sensitivity to x f_{x} \cdot = u_{1} (how fast x moves) \frac{d ( p _{1} + h u _{1} )}{d h} + height’s sensitivity to y f_{y} \cdot = u_{2} \frac{d ( p _{2} + h u _{2} )}{d h}$

Since $\frac{d ( p _{1} + h u _{1} )}{d h} = u_{1}$ and $\frac{d ( p _{2} + h u _{2} )}{d h} = u_{2}$ , this collapses to

$D_{\overset{u}{^}} f = f_{x} u_{1} + f_{y} u_{2}$

WHY the chain rule and not something else? Walking along $\overset{u}{^}$ moves $x$ and $y$ at once. The chain rule is precisely the tool that says "when several inputs move together, add up each input's contribution (its own slope times its own speed)." No other rule bookkeeps simultaneous motion.

PICTURE. The red step of length $h$ casts two shadows on the floor: an eastward shadow of length $h u_{1}$ and a northward shadow of length $h u_{2}$ . Height climbs $f_{x}$ per eastward metre and $f_{y}$ per northward metre. Add the two climbs — that's the equation.

Step 6 — Recognise a dot product; the gradient is born

WHAT. The expression $f_{x} u_{1} + f_{y} u_{2}$ is "multiply matching entries, then add". That pattern is the dot product of two vectors: $D_{\overset{u}{^}} f = (f_{x}, f_{y}) \cdot (u_{1}, u_{2})$ The first vector, $(f_{x}, f_{y})$ , depends only on the hill (not on which way we chose to walk). It deserves a name: the gradient $\nabla f$ .

WHY name it? Because it factors cleanly: hill-info ( $\nabla f$ ) times my-choice ( $\overset{u}{^}$ ). Once you separate "what the hill offers" from "what I choose", you can ask the hill's question — which choice is best? — without recomputing anything.

PICTURE. Two arrows drawn on the floor from $p$ : the fixed gradient $\nabla f = (f_{x}, f_{y})$ (yellow) and my chosen $\overset{u}{^}$ (red). The dot product is the yellow arrow's shadow cast onto the red direction, times $\overset{u}{^}$ 's length ( $= 1$ ).

Step 7 — Which direction climbs fastest? (the payoff)

WHAT. A dot product also equals $∣\nabla f ∣ ∣ \overset{u}{^} ∣ cos θ$ , where $∣\nabla f ∣$ is the gradient's length, $∣ \overset{u}{^} ∣ = 1$ , and $θ$ is the angle between $\nabla f$ and $\overset{u}{^}$ . So $D_{\overset{u}{^}} f = ∣\nabla f ∣ cos θ .$ The only thing we control is $θ$ . And $cos θ$ is largest when $θ = 0$ .

WHY $cos$ ? Because "how much of the gradient points along my walk" is a projection, and projection is $cos θ$ of the angle between them. It answers exactly: what fraction of the full uphill push lands in my chosen direction?

PICTURE. As the red $\overset{u}{^}$ swings around the fixed yellow $\nabla f$ , watch the climb-rate $∣\nabla f ∣ cos θ$ : maximal ( $+ ∣\nabla f ∣$ ) when they align, zero when perpendicular, most negative ( $- ∣\nabla f ∣$ ) when opposite.

Step 8 — The degenerate case: $\nabla f = 0$

WHAT. What if $f_{x} = 0$ and $f_{y} = 0$ ? Then $\nabla f$ is the zero vector: length $0$ , no direction. The formula gives $D_{\overset{u}{^}} f = 0 \cdot \overset{u}{^} = 0$ for every $\overset{u}{^}$ .

WHY show this? The contract: never leave a scenario unpainted. At a flat spot — a hilltop, a valley floor, or a saddle — there is no uphill. "Steepest ascent" has no answer because $cos θ$ gets multiplied by $∣\nabla f ∣ = 0$ . The reader must know the formula still works; it just returns "flat, all ways".

PICTURE. Three flat-gradient landscapes: a rounded peak, a bowl valley, and a saddle. On each, the gradient arrow has shrunk to a dot — every walking direction has zero initial slope. (These are exactly the points Lagrange multipliers and optimisation hunt for.)

The one-picture summary

Everything in one frame: the hill, the yellow gradient arrow lying on the floor pointing straight uphill, a red chosen direction, its shadow (the dot product) giving the climb-rate, and the level curve the gradient stabs at a right angle.

Recall Feynman: the whole walk in plain words

You're standing on a foggy hill. First we noticed you can only feel two slopes directly: how steep it is if you step due east ( $f_{x}$ ), and how steep due north ( $f_{y}$ ). But you might want to walk diagonally. So we asked: if you take a tiny step in some direction, how much do you climb? We turned that diagonal walk into a straight-line stroll measured by one number $h$ , and the chain rule told us the climb is just "east-slope times how-much-you-go-east, plus north-slope times how-much-you-go-north". That sum is secretly a dot product — line up two lists, multiply, add. One of those lists depends only on the hill; we bottled it and called it the gradient, an arrow lying on the ground. The climb in any direction is the gradient's shadow on that direction. Shadows are longest when you face the arrow head-on — so the steepest way up is simply along the gradient itself, and its length is how steep. Walk across the arrow and its shadow vanishes: you stay level. And if the arrow ever shrinks to a dot, you're on flat ground — a peak, a valley, or a saddle — where no direction climbs at all.

Recall

Why must $\overset{u}{^}$ have length 1 in $D_{\overset{u}{^}} f$ ? ::: To measure climb per unit distance; a longer vector would scale the answer by its length. What single object packs all directional slopes? ::: The gradient $\nabla f = (f_{x}, f_{y})$ . Which rule converts the diagonal walk into $f_{x} u_{1} + f_{y} u_{2}$ ? ::: The multivariable chain rule. Why is $\nabla f$ the steepest-ascent direction? ::: $D_{\overset{u}{^}} f = ∣\nabla f ∣ cos θ$ is maxed at $θ = 0$ , i.e. $\overset{u}{^}$ aligned with $\nabla f$ . What does $\nabla f = 0$ signal? ::: A first-order-flat point: peak, valley, or saddle — no steepest direction exists.

Connections

Partial derivatives — the two entries of $\nabla f$ (Step 2).
Directional derivative — the quantity we derived (Step 3).
Chain rule (multivariable) — the engine of Step 5.
Tangent plane and linearisation — the linear picture $\nabla f$ builds.
Level curves and surfaces — the curve $\nabla f$ meets at a right angle (Step 7).
Gradient descent — rides $- \nabla f$ downhill.
Lagrange multipliers — where $\nabla f = 0$ or $\nabla f = λ \nabla g$ matters (Step 8).