Backpropagation — chain rule, gradient computation: Foundations

5.6.8 · D1 · Coding › Machine Learning (Aerospace Applications) › Backpropagation — chain rule, gradient computation

Yeh page kuch bhi assume nahi karta. Agar tumne kabhi derivative, vector, ya chhote Greek letters nahi dekhe, toh yahan se shuru karo aur tum parent note ko line by line padh paoge.

0. Function kya hota hai (sab kuch ka atom)

Ek box imagine karo jisme ek arrow andar ja raha hai aur ek arrow bahar aa raha hai.

Figure — Backpropagation — chain rule, gradient computation

Yeh topic kyun zaroorat rakhta hai iska. Ek neural net in boxes ka ek stack hai. Poora parent note $L = f_{N} (\dots f_{2} (f_{1} (x)))$ likhta hai — woh sirf kai boxes ek line mein wired hain, ek ka output agले ka input banta hai.

1. Slope, aur hume derivative kyun chahiye

Tasveer: kisi bhi smooth curve mein itna zoom karo aur woh ek seedhi line jaisi dikhne lagti hai. Us chhoti line ki steepness hi derivative hai. Figure mein green tangent line dekho — uski steepness hi number $df / d x$ hai.

Yeh topic kyun zaroorat rakhta hai iska. Backprop mein har "local derivative" (jaise $σ^{'} (z_{1})$ ) inhi slopes mein se ek hai. Poora algorithm slopes multiply karna hai.

2. Partial derivative $\partial$ (ek saath kai knobs)

Tasveer: ek pahadi par khade ho. Poori taraf east chalte hue ek slope hai; poori taraf north chalte hue doosra. Har direction usi pahadi ka ek partial derivative hai.

Yeh topic kyun zaroorat rakhta hai iska. Ek net mein millions of weights $w$ hote hain. Hum har ek ke liye $\partial L / \partial w$ chahte hain — error ki sensitivity us single knob ke liye jabki baaki frozen hain. Saare partials ki woh list gradient hai.

3. Gradient $\nabla$ (slopes ka poora bundle)

Tasveer: pahadi par, east-slope aur north-slope ko ek single arrow mein ikatha karo. Woh arrow seedha upar pahadi ki taraf aim karta hai; uski length batati hai kitna steep. Error reduce karne ke liye tum opposite direction mein step karte ho — woh hai Gradient Descent.

Yeh topic kyun zaroorat rakhta hai iska. Backprop ka poora kaam is gradient ko saste mein compute karna hai. Gradient Descent phir isse consume karta hai.

4. Chain rule (slopes kyun multiply hote hain)

Tasveer: do boxes ek line mein. Ek tiny push $d x$ andar jaata hai, pehle box ki slope se scale hota hai, phir doosre box ki slope se scale hota hai. Do multiplications, order mein.

Fan-in case (sum). Agar $x$ , $L$ ko kai routes $y_{1}, \dots, y_{k}$ se affect karta hai, toh har route apna multiply kiya hua slope contribute karta hai aur hum unhe add karte hain: $\frac{\partial L}{\partial x} = \sum_{i = 1}^{k} \frac{\partial L}{\partial y _{i}} \frac{\partial y _{i}}{\partial x} .$

Sum kyun? $x$ mein change har wire mein ripple karta hai jise woh feed karta hai; total effect woh saare effects ek saath hain. Yeh multivariable chain rule hai aur isi liye parent warn karta hai "fan-in node par, saare paths par sum karo."

Yeh topic kyun zaroorat rakhta hai iska. Backprop = chain rule computation graph mein backward apply kiya gaya, cached numbers reuse karte hue.

5. Vectors, matrices, aur transpose $⊤$

Tasveer: ek matrix ek wiring board hai — row $i$ , column $j$ entry $W_{ij}$ input $j$ se output $i$ tak wire ki strength hai.

Yeh topic kyun zaroorat rakhta hai iska. Forward pass signals $W$ se bhejta hai. Error ko backward bhejna $W^{⊤}$ use karta hai — kyunki linear map $W a$ ka slope $a$ ke respect mein exactly $W^{⊤}$ hai. Parent ki sabse badi dar wali mistake ("backprop ko inverse chahiye") isse fix hoti hai: yeh ek transpose hai, inverse nahi. Wires reverse karna ≠ unhe undo karna.

6. Elementwise product $⊙$

Tasveer: numbers ke do parallel columns; same row par har pair ko multiply karo.

Yeh topic kyun zaroorat rakhta hai iska. Har unit apni incoming error ko apni slope $σ^{'} (z)$ se scale karta hai. Same-slot scaling = $⊙$ . Wahi parent ki boxed backprop equation mein $⊙ σ^{'} (z)$ hai.

7. Activation $σ$ aur uski slope $σ^{'}$

Yeh topic kyun zaroorat rakhta hai iska. Agar $σ^{'} (z) \approx 0$ ("saturated"/sleepy unit) hai, toh almost koi error pass back nahi hota — yahi Vanishing Gradients ka seed hai. Agar hum $σ^{'}$ factor bilkul bhool jaayein, toh nonlinear units mein gradients simply galat hain.

8. Parent ke baaki alphabet (quick glossary)

Symbol	Simple words	Tasveer
$x$	net ka input	pehle box mein jaata arrow
$w, b$	ek weight (wire strength) aur ek bias (ek constant shove)	dial aur ek nudge
$z^{(ℓ)}$	raw sum $W a + b$ bend se pehle, layer $ℓ$ mein	pre-activation
$a^{(ℓ)}$	bend $σ (z)$ ke baad ka output	activation
$\overset{y}{^}$	net ki prediction (output)	bahar jaata aakhri arrow
$t$	woh true target jo hum chahte the	bullseye
$L$	loss = hum kitne galat hain	ek number, zyada = bura
$δ^{(ℓ)}$	layer $ℓ$ ki pre-activation par error signal, $\partial L / \partial z^{(ℓ)}$	"kitna blame yahan girta hai"
superscript $(ℓ)$	kaunsi layer	ek building mein floor number

$δ$ (delta) sabse zyada kyun matter karta hai. Yeh reusable quantity hai: ise ek baar per layer compute karo, aur us layer ka weight-gradient aur agली layer ka $δ$ dono isse nikal aate hain. $δ$ reuse karna hi wajah hai ki backprop sasta hai — Automatic Differentiation idea disguise mein.

Prerequisite map

Baayein taraf har foundation backprop node ko feed karta hai; backprop phir training loop ko feed karta hai.

Equipment checklist

Self-test: kya tum bina dekhe har ek ka jawab de sakte ho?

f_{2} (f_{1} (x))

tumhe pehle kya karne ko kehta hai?

x

f_{1}

se guzaro, phir woh result

f_{2}

mein feed karo (andar se bahar).

Ek word mein, derivative kya hai?

Ek slope (rate of change).

Curly

\partial

kya signal karta hai jo plain

d

nahi karta?

Doosre knobs frozen hain jabki hum sirf ek ko wiggle kar rahe hain.

Gradient

\nabla L

kya hai, seedha seedha?

Saare partial derivatives ki list, ek slope per knob.

Chain rule paanch words mein?

Slopes along a chain multiply.

Jab ek variable kai paths feed karta hai, path contributions multiply karein ya add?

Add karo (multivariable chain rule).

W^{⊤}

physically kya karta hai aur

W^{- 1}

kyun nahi?

Wiring reverse karta hai;

W a

ka differentiation transpose deta hai, inverse nahi.

u ⊙ v

kya compute karta hai?

Slot-by-slot (elementwise) products, koi mixing nahi.

Nonlinearity cross karte waqt

σ^{'} (z)

factor kyun rakhte hain?

Har unit apni incoming error ko apni activation slope se scale karta hai; isse skip karna galat hai (aur near-zero slope vanishing gradients cause karta hai).

δ^{(ℓ)}

kya hai aur ise naam dena kyun worth it hai?

Ek layer ki pre-activation par error

\partial L / \partial z^{(ℓ)}

; ise cache karne se us layer mein (aur agला

δ

mein) har gradient isse reuse kar sakta hai.

Connections

5.6.08 Backpropagation — chain rule, gradient computation (Hinglish) — parent, Hinglish mein.
Chain Rule — Section 4 mein bana slopes-ka-multiplication engine.
Gradient Descent — woh gradient consume karta hai jise hum ne Section 3 mein naam dena seekha.
Computational Graphs — Sections 0 aur 8 se wired-boxes ki tasveer.
Activation Functions — Section 7 ka bendy box $σ$ .
Vanishing Gradients — kya hota hai jab $σ^{'} (z) \approx 0$ .
Automatic Differentiation — reuse- $δ$ idea jo machines ne generalize kiya.
Neural Network Surrogate Models (CFD) — jahan yeh sab aerospace mein kaam aata hai.