Gradients and directional derivatives — Hinglish notes

1.2.4 · AI-ML › Calculus & Optimization Basics

KIYA hai partial derivative (building block)

Baaki ko constant kyun maante hain? Kyunki partial derivative change ko ek coordinate axis ke saath measure karta hai. East ki taraf chalna tumhe north slope ke baare mein kuch nahi batata, isliye hum north ko freeze kar dete hain.

KIYA hai gradient

KAISE derive karte hain directional derivative

Goal: $f$ ki change ki rate agar hum point $a$ se unit direction $u$ mein move karein (jahan $∥ u ∥ = 1$ ).

Us direction ke saath ek 1-D slice define karo: $g (t) = f (a + t u)$ Directional derivative bas $g^{'} (0)$ hai — us slice ka ordinary 1-D derivative.

Chain rule apply karo. Maano $x (t) = a + t u$ , toh $x_{i} (t) = a_{i} + t u_{i}$ aur $\frac{d x _{i}}{d t} = u_{i}$ : $g^{'} (t) = \sum_{i = 1}^{n} \frac{\partial f}{\partial x _{i}} \frac{d x _{i}}{d t} = \sum_{i = 1}^{n} \frac{\partial f}{\partial x _{i}} u_{i}$

Yeh sum bilkul ek dot product hai:

KYUN gradient steepest uphill point karta hai

Dot product ko geometric form mein likho: $D_{u} f = \nabla f \cdot u = ∥\nabla f ∥ ∥ u ∥ cos θ = ∥\nabla f ∥ cos θ$ jahan $θ$ , $u$ aur $\nabla f$ ke beech ka angle hai (aur $∥ u ∥ = 1$ ).

Yeh sabse bada tab hota hai jab $cos θ = 1$ , yaani $θ = 0$ ⇒ $\nabla f$ ke saath chalo. → steepest ascent, value $= ∥\nabla f ∥$ .
Sabse chhota (sabse zyada negative) jab $θ = π$ ⇒ $\nabla f$ ke opposite chalo. → steepest descent, isliye gradient descent $- \nabla f$ use karta hai.
Zero jab $θ = π /2$ ⇒ $\nabla f$ ke perpendicular chalo. → tum level set / contour ke saath chal rahe ho, height unchanged. Toh gradient contour lines ke perpendicular hota hai.

Figure — Gradients and directional derivatives

Worked examples

Common mistakes (steel-manned)

Forecast-then-verify

Flashcards

What is the directional derivative

D_{u} f

in dot-product form?

D_{u} f = \nabla f \cdot u

with

∥ u ∥ = 1

Why must

u

be a unit vector?

Taaki result slope ko per unit distance measure kare, vector ki length se scale na ho.

In which direction does

\nabla f

point?

Steepest ascent ki direction mein; iska magnitude steepest slope hai.

What is the relationship between

\nabla f

and contour lines?

Yeh level sets ke perpendicular (orthogonal) hota hai, kyunki contour ke saath

D_{u} f = 0

Why does gradient descent use

- \nabla f

- \nabla f

steepest descent ki direction hai (

cos θ = - 1

f

ki sabse tezi se decrease.

Is the gradient a scalar or a vector?

Ek vector jisme har input variable ke liye ek component hota hai.

Derivation: how do we get

D_{u} f = \nabla f \cdot u

g (t) = f (a + t u)

set karo, chain rule apply karo;

g^{'} (0) = \sum_{i} \frac{\partial f}{\partial x _{i}} u_{i} = \nabla f \cdot u

Max value of

D_{u} f

over all unit

u

∥\nabla f ∥

, tab milta hai jab

u

\nabla f

ke saath align ho.

Recall Feynman: 12-saal ke bachche ko samjhao

Tum ek ulte-seedhe pahaad par ho. Jahan bhi khade ho, ek direction sabse tezi se upar jaati hai — jaise slide ka sabse steep hissa, par upar ki taraf. Gradient bilkul us steepest-up direction mein point karta ek arrow hai, aur arrow ki lambai batati hai kitna steep hai. Agar tum bottom tak pahunchna chahte ho (model train karna), toh hamesha arrow ke ulti direction mein chalo. Aur agar tum arrow ke sideways chalo, toh tum same height par rehte ho — yeh pahaad par ek flat ring ke saath chalna hai.

Connections

Partial derivatives — woh components jo $\nabla f$ banate hain.
Chain rule — directional derivative derive karne mein use hota hai.
Gradient descent — $- \nabla f$ ke saath chalta hai.
Level sets and contours — gradient unke orthogonal hota hai.
Dot product and projections — $\nabla f \cdot u$ ke peeche ki geometry.
Jacobian and Hessian — higher-order generalizations.