Visual walkthrough — Gradient as direction of steepest ascent

4.4.10 · D2 · Maths › Multivariable Calculus › Gradient as direction of steepest ascent

Hum assume karte hain ki tum bas yeh jaante ho: ek function $f(x,y)$ zameen par har point ke liye ek height deta hai. Baaki sab — slopes, dot products, "fastest" — hum chalte chalte earn karenge.

---

Step 1 — Ek hill ek height map hoti hai

KYA HAI. Ek landscape imagine karo. Flat zameen par har jagah ka ek address $(x,y)$ hai: kitna east ($x$) aur kitna north ($y$). Function $f(x,y)$ tumhe ek number wapas deta hai — us jagah ki zameen ki height.

KYUN. Pehle "steepest" ki baat karne se pehle, humein koi cheez chahiye jisme steepness ho. Height map wahi cheez hai. Hum isse do tareekon se draw karte hain: ek 3-D surface ke roop mein, aur ek contour map ke roop mein — woh flat top-down view jisme hum same height waale saare points ko ek loop se jodte hain, bilkul hiking map ki tarah.

PICTURE. Left panel mein 3-D hill $f=x^2+y^2$ hai (ek bowl). Right panel mein wahi hill seedha upar se dekhi gayi hai: har pale-blue loop ek level curve hai, yani aisi points ka set jo sab ek hi fixed height par hain. Loops paas paas = steep zameen; loops door door = gentle zameen.

---

Step 2 — "Ek direction kya hoti hai?" — unit arrow

KYA HAI. Ek point par khade hokar, tum infinitely many directions mein chal sakte ho. Hum har direction ko ek chhota arrow $\mathbf{u}=(u_1,u_2)$ se label karte hain. Yahan $u_1$ batata hai ki step ka kitna hissa east ki taraf gaya, $u_2$ kitna north ki taraf.

KYUN. Hum directions ko fair tarike se compare karne wale hain. Agar ek arrow lamba hota aur doosra chhota, toh lamba arrow height zyada change karta dikhta — sirf isliye ki woh bada step hai — na ki isliye ki woh direction genuinely steeper hai. Toh hum force karte hain ki har direction-arrow ki same length ho, exactly 1. Woh length $\Vert \mathbf{u}\Vert$ likhi jaati hai.

$\Vert \mathbf{u}\Vert =\underset{\text{Pythagoras se length}}{\underbrace{\sqrt{u_1^2+u_2^2}}}=1$

Yahan $u_1^2+u_2^2$ squared length hai (east-leg${}^2$ + north-leg${}^2$, ek right triangle), aur square root isse wapas length mein convert karta hai. Isse $1$ set karne par har arrow radius one ke circle par aa jaata hai.

PICTURE. Saare candidate direction-arrows point se fan out karte hain; unki tips yellow unit circle trace karti hain. Har arrow ek fair, equal-length step hai.

---

Step 3 — Ek arrow ke saath height kitni tezi se badlti hai?

KYA HAI. Pehle, ek shorthand: starting point ko har baar pair $(x,y)$ likhne ki jagah, hum un do numbers ko ek single position vector mein bundle karte hain aur isse ==$\mathbf{x}=(x,y)$== kehte hain — ek bold letter jo us point ko represent karta hai jahan tum khade ho. Ab ek direction $\mathbf{u}$ chuno. Size $h$ ka ek chhota step us direction mein lo, toh tum new point $\mathbf{x}+h\mathbf{u}$ par pahunch jaate ho (start, plus direction arrow ka $h$ hissa). Dekho ki height kitni badi, phir divide karo kitna chale usse. Step ko zero tak shrink karo. Woh limiting ratio hi directional derivative $D_{\mathbf{u}}f$ hai.

$D_{\mathbf{u}}f={\lim }_{h\to 0}\frac{\stackrel{\text{step ke baad height}}{\overbrace{f(\mathbf{x}+h\mathbf{u})}}-\stackrel{\text{abhi height}}{\overbrace{f(\mathbf{x})}}}{\underset{\text{chali gayi distance}}{\underbrace{h}}}$

Yahan $\mathbf{x}$ woh point hai jahan se shuru karte ho, $\mathbf{x}+h\mathbf{u}$ woh point hai $h$ length ka step direction $\mathbf{u}$ mein lene ke baad. Numerator height mein change hai; denominator $h$ chal kar ki gayi distance hai. Rise over run — ek slope.

KYUN limit? Kyunki hill curve karti hai. Bade step par slope constant nahi hoti, toh bada-step ratio ek average hoga, na ki yahaan ka actual slope. $h\to 0$ shrink karna zoom in karta hai jab tak zameen flat nahi lagti aur ratio ek honest number par settle ho jaata hai: us exact jagah us direction mein steepness.

PICTURE. Hum hill ko arrow $\mathbf{u}$ wali vertical plane se slice karte hain. Cut edge ek 1-D curve hai; $D_{\mathbf{u}}f$ us point par us curve ka slope hai (pink tangent line).

---

Step 4 — Infinitely many limits lena band karo: chain rule

KYA HAI. Infinitely many arrows mein se har ek ke liye woh limit fresh calculate karna hopeless hai. Iske bajay, direction $\mathbf{u}$ mein seedhi line par chalo aur us par height ko $g(h)$ kaho. Starting point $\mathbf{x}$ ko components mein ==$(x_0,y_0)$== likho, toh $x_0$ uski east-coordinate hai aur $y_0$ uski north-coordinate. Direction $\mathbf{u}=(u_1,u_2)$ mein $h$ distance step lene ke baad, tumhari east-coordinate $x_0+h{u}_1$ hai aur north-coordinate $y_0+h{u}_2$:

$g(h)=f(\underset{x(h)}{\underbrace{x_0+h{u}_1}},\underset{y(h)}{\underbrace{y_0+h{u}_2}})$

Ab $g$ ek ordinary ek-input function of $h$ hai, aur $D_{\mathbf{u}}f=g^{\prime }(0)$ hai.

KYUN. Jaise $h$ badta hai, dono $x$ aur $y$ drift karte hain. Multivariable Chain Rule kehta hai ki height mein total change = $x$ ke move karne se aaya change, plus $y$ ke move karne se aaya change:

$g^{\prime }(h)=\underset{\text{f kitna east tilt karta hai}}{\underbrace{\frac{\partial f}{\partial x}}}\cdot \underset{=u_1}{\underbrace{\frac{dx}{dh}}}+\underset{\text{f kitna north tilt karta hai}}{\underbrace{\frac{\partial f}{\partial y}}}\cdot \underset{=u_2}{\underbrace{\frac{dy}{dh}}}$

Yahan $\frac{\partial f}{\partial x}$ (ek partial derivative, $f_x$ likha jaata hai) woh slope hai agar sirf east mein move karte; $\frac{dx}{dh}=u_1$ isliye kyunki $x$ har unit of $h$ ke liye $u_1$ se badhta hai. North ke liye same story. $h=0$ set karne par sab kuch starting point $(x_0,y_0)$ par evaluate ho jaata hai:

$\boxed{D_{\mathbf{u}}f=f_x{u}_1+f_y{u}_2}$

PICTURE. $\mathbf{u}$ ke saath total slope do known slopes se banta hai: east slope $f_x$ ko multiply karo step ka kitna hissa eastward hai ($u_1$) se, plus north slope $f_y$ ko $u_2$ se.

---

Step 5 — Dot product pehchano

KYA HAI. Numbers ki dono lists ko align karo. Partials ko ek arrow mein collect karo, gradient $\nabla f=(f_x,f_y)$. Phir $f_x{u}_1+f_y{u}_2$ bilkul wahi hai jo $\nabla f$ ka $\mathbf{u}$ ke saath dot product hai:

$D_{\mathbf{u}}f=\underset{\nabla f}{\underbrace{(f_x,f_y)}}\cdot \underset{\mathbf{u}}{\underbrace{(u_1,u_2)}}=\nabla f\cdot \mathbf{u}$

KYUN matter karta hai. Dot product "matching components multiply karo, add karo" hai. Lekin iska ek doosra, geometric face bhi hai — aur wahi face agले step mein poora answer unlock karta hai.

PICTURE. Point se do arrows: ==pale-yellow gradient $\nabla f$ aur unit circle par blue direction $\mathbf{u}$==, unke beech angle mark kiya hua.

---

Step 6 — Dot product ka geometric face

KYA HAI. Pehle us quantity ko naam do jo hum abhi use karne wale hain: ==$\theta$ ko do arrows $\nabla f$ aur $\mathbf{u}$ ke beech ka angle maano== — kitna tum ek arrow ko rotate karte toh woh doosre ke saath line up hota (yeh wahi angle hai jo Step 5 picture mein draw hua hai). Kisi bhi do arrows ke liye, dot product unki lengths ka product times us angle ka cosine bhi hota hai:

$\nabla f\cdot \mathbf{u}=\underset{\nabla f\text{ ki length}}{\underbrace{\Vert \nabla f\Vert }}\underset{=1}{\underbrace{\Vert \mathbf{u}\Vert }}\underset{\text{alignment, }\theta =\angle (\nabla f,\mathbf{u})}{\underbrace{\cos \theta }}=\Vert \nabla f\Vert \cos \theta$

Kyunki $\Vert \mathbf{u}\Vert =1$ hai (Step 2), yeh drop out ho jaata hai. Toh

$\boxed{D_{\mathbf{u}}f=\Vert \nabla f\Vert \cos \theta }$

KYUN cosine? Cosine alignment measure karta hai. Jab do arrows same direction mein point karte hain ($\theta =0$), $\cos \theta =1$ (full agreement). Right angle par ($\theta =\frac{\pi }{2}$), $\cos \theta =0$ (koi agreement nahi). Opposite ($\theta =\pi$), $\cos \theta =-1$. Yeh exactly woh "yeh do arrows kitna agree karte hain?" dial hai jo humein chahiye.

PICTURE. Ek single dial: jaise $\mathbf{u}$ circle ke aas paas swing karta hai, uska aur fixed gradient ke beech angle $\theta$ badlta hai, aur value $\Vert \nabla f\Vert \cos \theta$ upar neeche hoti hai. Sirf ek cheez change kar sakte ho $\theta$; $\Vert \nabla f\Vert$ is point par hill ka fixed property hai.

---

Step 7 — Kaun sa direction jeetta hai? (payoff)

KYA HAI. Hum woh $\mathbf{u}$ chahte hain jo $D_{\mathbf{u}}f=\Vert \nabla f\Vert \cos \theta$ ko as large as possible banaye. Sirf ek free knob hai $\theta$, aur $\cos \theta$ sabse bada hota hai — $1$ ke barabar — jab $\theta =0$ hota hai. Toh winning direction woh hai jo $\nabla f$ ke same taraf point kare.

${\mathbf{u}}^{\ast }=\frac{\nabla f}{\Vert \nabla f\Vert },{\max }_{\mathbf{u}}{D}_{\mathbf{u}}f=\Vert \nabla f\Vert \cdot 1=\Vert \nabla f\Vert$

Yahan $\nabla f$ ko apni length $\Vert \nabla f\Vert$ se divide karne par woh length $1$ tak shrink ho jaata hai — ek unit arrow gradient ki taraf point karta hua. Maximum rate bas $\Vert \nabla f\Vert$ hi hoti hai.

KYUN yeh sab settle kar deta hai. Koi arithmetic nahi, koi calculus of variations nahi — cosine simply $1$ se zyada nahi ho sakta, aur woh $1$ hit karta hai sirf $\nabla f$ ke along. Bas wahi ek fact hi "steepest ascent = gradient direction" hai.

PICTURE. Directions ka compass jo $D_{\mathbf{u}}f$ ke hisaab se colour-graded hai: bilkul $\nabla f$ ke along sabse bright (upar), usse opposite sabse dark (neeche).

---

Step 8 — Baaki cases (kuch bhi uncovered nahi)

KYA HAI. Wahi formula $D_{\mathbf{u}}f=\Vert \nabla f\Vert \cos \theta$ ko doosre special angles par padho, aur us ek jagah par jo formula toot jaata hai.

$\theta$	$\cos \theta$	$D_{\mathbf{u}}f$	kya hota hai
$0$	$+1$	$+\Vert \nabla f\Vert$	sabse steep ascent (gradient ke upar)
$\frac{\pi }{2}$	$0$	$0$	height nahi badlti → level curve ke saath chalna
$\pi$	$-1$	$-\Vert \nabla f\Vert$	sabse steep descent ($-\nabla f$; yeh hai Gradient Descent (Machine Learning))

Perpendicular fact. $\theta =\frac{\pi }{2}$ par height nahi badlti, toh woh direction hi level curve ki tangent hai. Kyunki woh $\nabla f$ se right angle par hoti hai, gradient har level curve se perpendicular hota hai. Isliye contour map par uphill arrow hamesha contours ko $90^{\circ }$ par cross karta hai.

Degenerate case: $\nabla f=0$. Agar dono partials vanish ho jaayein, toh $\Vert \nabla f\Vert =0$, isliye $D_{\mathbf{u}}f=0$ har direction ke liye — zameen momentarily flat hai (ek peak, ek valley floor, ya ek saddle). Koi steepest direction nahi hai, aur formula ${\mathbf{u}}^{\ast }=\nabla f/\Vert \nabla f\Vert$ undefined hai kyunki zero se divide nahi kar sakte. Yeh woh ek input hai jahan "upar point karo" ka koi jawab nahi hai, aur maths honestly ek dene se mana kar deta hai.

PICTURE. Left: gradient pale-blue contours ko right angles par cross karta hua. Right: ek flat spot jahan $\nabla f=0$ — arrow ek dot mein collapse ho jaata hai aur saari directions equally (un)steep hain.

---

Ek-picture summary

KYA HAI. Ek figure poori chain ko tie karta hai: gradient arrow wala ek contour map, unit directions ka fan jo unke $D_{\mathbf{u}}f$ se coloured hai, aur woh chhota "$\cos \theta$ dial" jo winner decide karta hai.

Ek saanss mein logic: directional derivative $=\nabla f\cdot \mathbf{u}$ (chain rule) $=\Vert \nabla f\Vert \cos \theta$ (dot product ki geometry) $\le \Vert \nabla f\Vert$, equality sirf tab jab $\mathbf{u}$ $\nabla f$ ke saath align ho.

Recall Feynman: kisi dost ko fog mein batao

Tum ek foggy hill par ho aur sirf paon ke neeche slope feel kar sakte ho. Sochte ho: kaun sa raasta sabse steep upar hai? Har direction try karna aur slope measure karna (yahi directional derivative hai) forever lag jaayega. Toh tumhe ek shortcut dikha: kisi bhi direction mein slope = east-slope times kitna-east-jao plus north-slope times kitna-north-jao. Do slopes ko ek arrow mein bundle karo — gradient — aur woh sum exactly hai "tumhara walking arrow, gradient arrow se kitna agree karta hai". Agreement cosine se measure hota hai, jo $1$ par max hota hai jab dono arrows same taraf point karein. Toh sabse steep raasta upar sirf gradient ke saath hai, aur kitna steep hai woh gradient ki length ke barabar hai. Right angle par sideways chalo aur cosine $0$ hai — zameen flat rehti hai, isliye tum ab ek contour line par stroll kar rahe ho. Poora ghoom jaao aur tumhe sabse steep raasta neeche milega. Aur agar gradient zero arrow hai, toh zameen har direction mein flat hai aur koi "upar" hi nahi hai.

---

Connections

• Gradient as direction of steepest ascent
• Directional Derivative
• Partial Derivatives
• Multivariable Chain Rule
• Dot Product and Angle
• Level Curves and Contour Maps
• Gradient Descent (Machine Learning)
• Tangent Plane and Linearization