Question bank — Backpropagation — chain rule, gradient computation

False — backprop sirf gradients compute karta hai; Gradient Descent ek alag step hai jo un gradients ko weights update karne ke liye use karta hai. Ek supply karta hai, doosra consume.

True — gradient $\partial L/\partial w$ tabhi meaningful hai jab $L$ ek single number ho; ek vector output ko pehle scalar loss mein reduce karna padta hai tab koi backward sweep shuru ho sakti hai.

False — backprop ko forward pass se cached activations $a^{(\ell -1)}$ aur pre-activations $z^{(\ell )}$ chahiye hote hain, kyunki local derivatives jaise ${\sigma }^{\prime }(z)$ aur input factor unhi par depend karte hain.

True — cost computation graph mein operations ki sankhya ke saath scale hoti hai, isliye depth ke saath linearly badhti hai, individual weights ki sankhya ke saath nahi.

True — error nonlinearity cross karte waqt ${\sigma }^{\prime }(z)=0$ se multiply hota hai, isliye wo path upstream weight gradients mein zero contribute karta hai (yahi Vanishing Gradients ki ek jad hai).

True — ye graph par symbolically apply kiya gaya chain rule hai, isliye (floating-point tak) ye exact hai, unlike finite-difference estimates jo truncation error carry karte hain.

True — kyunki $z=Wa+b$ se $\partial z/\partial b=1$ milta hai, isliye bias raw $\delta$ inherit karta hai bina kisi input factor ke.

False — automatic differentiation in hi rules ko arbitrarily large graphs par mechanically apply karta hai; haath se derivation bas yahi dikhata hai jo machine karta hai.

Ye downstream error drop kar deta hai: $\partial L/\partial w_2={\delta }_2{a}_1$, na ki sirf $a_1$. Akela $a_1$ sirf local derivative $\partial z_2/\partial w_2$ hai; baaki network ka blame ${\delta }_2$ isse multiply hota hai.

Backprop transpose $W^{\top }$ use karta hai, inverse nahi. Linear map $Wa$ ka $a$ ke respect mein derivative $W^{\top }$ hai; differentiation transpose karta hai, kabhi invert nahi karta (aur $W$ ka square hona bhi zaroori nahi).

Ye $\odot {\sigma }^{\prime }(z^{(\ell )})$ factor bhool jaata hai. Hidden layers ek nonlinearity se guzarte hain, isliye unka incoming error har unit ke activation slope se scale hona chahiye; sirf linear output layer ise skip karta hai.

Tumhe saare outgoing paths ke contributions sum karne chahiye (multivariable Chain Rule). Har path ek independent tarika hai jisse node loss ko influence karta hai, aur total sensitivity unka sum hai.

MSE $L=\frac{1}{2}(\hat{y}-t{)}^2$ ke liye seed $\hat{y}-t$ hai, yaani error, na ki $\hat{y}$. Target $t$ miss karne se poora downstream gradient galat ho jaata hai.

Har local derivative ko scratch se recompute karna efficiency destroy kar dega; $O(1)$-forward-pass ka poora advantage activations cache karne se aata hai taaki har backward step sasta ho.

Orientation galat hai — ye outer product ${\delta }^{(\ell )}{a}^{(\ell -1)\top }$ hai, jo $W^{(\ell )}$ jaisi shaped matrix deta hai (outputs × inputs). Transpose swap karne se shape galat ho jaati hai.

Kyunki chain rule derivative ko local slopes ke product mein factor karta hai, aur ek single backward sweep har cached intermediate reuse karke saare weight gradients ek saath fill kar deta hai.

Loss door ke end par ek scalar hai; us ek scalar se backward sweep karna har shared sub-derivative reuse karta hai, jabki forward accumulation har input weight ke liye same downstream factors recompute karta.

Hum differentiate kar rahe hain, isliye activation ka slope (${\sigma }^{\prime }$) chahiye, jo batata hai ki $z$ mein thoda change $a$ ko kitna change karta hai — $\sigma (z)$ ki value to forward pass ne use ki thi, sensitivity nahi.

Kyunki har layer ka pre-activation identical linear form $z=Wa+b$ rakhta hai, isliye $\partial z/\partial W$ hamesha input $a$ pick out karta hai — recursion bas har depth par is local structure ko repeat karta hai.

Unke flat regions mein ${\sigma }^{\prime }(z)\approx 0$ hota hai, isliye sahi math bhi error ko near-zero se multiply karta hai, kai layers mein gradients shrink hote hain — yahi Vanishing Gradients ka mechanism hai.

Ek CFD surrogate fit karne ka matlab hai huge datasets par millions of weights tune karna; sirf backprop ka one-sweep gradient hi har training step ko itna affordable banata hai ki convergence ho sake.

Wahan slope undefined hai (ek kink), isliye implementations ek subgradient choose karte hain — usually ${\sigma }^{\prime }(0)=0$ (kabhi kabhi $1$); ye ek measure-zero case hai jo training ko rarely affect karta hai.

Nahi — gradient har weight ki apni location par aane wali error aur input par depend karta hai, jo generally alag hote hain, isliye equal weights ke gradients bahut alag ho sakte hain.

Uska ${\sigma }^{\prime }(z)=0$ saare inputs par hai, isliye ye backward zero error pass karta hai aur hamesha ke liye zero weight gradient receive karta hai — ye stuck hai aur apne aap recover nahi kar sakta.

Haan — uska gradient exactly $\delta$ hai, us unit par raw error; constant input ka bas matlab hai ki ise multiply karne ke liye koi extra input factor nahi hai.

Ye all-zero gradients return karta hai — ek sahi answer matlab descent move nahi karega, jo minima, maxima, saddle points, ya dead-flat plateaus sab par hota hai.

Zero, weight ki apni value ya input ki parwah kiye bina, kyunki $\partial L/\partial w=\delta \cdot a$ aur $\delta =0$ product ko nullify kar deta hai.