1.2.5
Intuition Badi picture (WHY)Ek single-input, single-output function f ( x ) f ′ ( x ) vector andar lete hain aur vector bahar dete hain: f : R n → R m table hai, har ek (output, input) pair ke liye. Woh table Jacobian matrix hai. Yeh kisi bhi point ke paas ek vector-valued function ki sabse achhi linear approximation hai — f ′ ( x )
Definition Jacobian matrixf : R n → R m f ( x ) = f 1 ( x 1 , … , x n ) ⋮ f m ( x 1 , … , x n ) , Jacobian J J f ∂ x ∂ f m × n J ij = ∂ x j ∂ f i . i f i j x j
J = ∂ x 1 ∂ f 1 ⋮ ∂ x 1 ∂ f m ⋯ ⋱ ⋯ ∂ x n ∂ f 1 ⋮ ∂ x n ∂ f m
Intuition Shape mnemonic — "rows are outputs, columns are inputs"Jacobian mein ==m hain (ek har output ke liye) aur n J ∈ R m × n Δ x ∈ R n Δ f ∈ R m ( m × n ) ( n × 1 ) = ( m × 1 )
Intuition Derivation ideaf x woh number hai jo linear approximation ko kaam karta banata hai:
f ( x + h ) ≈ f ( x ) + f ′ ( x ) h . J f ( x + h ) ≈ f ( x ) + J h .
Step 1 — Ek output ek waqt dekho. Component f i
f i ( x + h ) ≈ f i ( x ) + j = 1 ∑ n ∂ x j ∂ f i h j . Yeh step kyun? Yeh sirf multivariable Taylor / total-differential rule hai: har input x j h j f i ∂ f i / ∂ x j
Step 2 — Sum ko dot product samjho.
j ∑ ∂ x j ∂ f i h j = row i [ ∂ x 1 ∂ f i ⋯ ∂ x n ∂ f i ] h = ( ∇ f i ) ⊤ h . Yeh step kyun? h j h f i
Step 3 — Saare m Yeh har output ke liye simultaneously karte hue:
f ( x + h ) ≈ f ( x ) + J h , J = ( ∇ f 1 ) ⊤ ⋮ ( ∇ f m ) ⊤ . Yeh step kyun? Row-approximations ko stack karna ek matrix equation ke barabar hai. Jis matrix ki rows gradients hain wahi Jacobian hai. ∎
Worked example Example 1 — ek2 → 2 f ( x , y ) = [ x 2 y x + sin y ] J ( 1 , 0 )
Partials:
∂ f 1 / ∂ x = 2 x y ∂ f 1 / ∂ y = x 2 ∂ f 2 / ∂ x = 1 ∂ f 2 / ∂ y = cos y
J ( x , y ) = [ 2 x y 1 x 2 cos y ] , J ( 1 , 0 ) = [ 0 1 1 1 ] . Har step kyun? Har entry "output i j ( 1 , 0 )
Worked example Example 2 — Jacobians se chain rule (backprop se match karta hai)Maano z = W x W ∈ R m × n a = σ ( z )
J z = ∂ x ∂ z = W Kyun? z i = ∑ j W ij x j ∂ z i / ∂ x j = W ij J σ = ∂ z ∂ a = diag ( σ ′ ( z 1 ) , … , σ ′ ( z m ) ) Kyun? a i = σ ( z i ) z i Poora map x ↦ a J = J σ J z = diag ( σ ′ ( z )) W .
Yeh kyun matter karta hai: Jacobians ka yeh product hai hi neural net ka forward-mode differentiation.
Worked example Example 3 — polar → Cartesian & determinantf ( r , θ ) = ( r cos θ , r sin θ ) J = [ cos θ sin θ − r sin θ r cos θ ] , det J = r cos 2 θ + r sin 2 θ = r . Determinant kyun? Square Jacobian ke liye, ∣ det J ∣ local area/volume scaling factor hai — ek chhota patch f r d r d θ
Common mistake "Jacobian aur gradient same cheez hain."Kyun sahi lagta hai: scalar function ke liye dono mein same numbers hote hain. Fix: gradient ∇ f column vector hai; m = 1 transpose hai (ek row). Isse bhi important, "gradient" tabhi meaningful hai jab output scalar ho (m = 1 m > 1
n × m Kyun sahi lagta hai: log aadat se pehle inputs likhte hain. Fix: yeh m × n rows = outputs, columns = inputs — kyunki isko R n → R m J h h ∈ R n
Common mistake "Jacobians ke liye chain ruleJ f J g Kyun sahi lagta hai: scalar chain rule d x d h = d g d h d x d g Fix: matrix multiplication commutative NAHI hai. h = g ∘ f outer function ka Jacobian left mein lagao: J h = J g J f
det J Kyun sahi lagta hai: hum har waqt determinants lete hain. Fix: determinant ke liye square matrix chahiye, yaani m = n J
Recall Feynman: ek 12-saal ke bacche ko explain karoEk aisi machine socho jo input mein kuch dials leti hai aur output mein kuch bulbs jalati hai. Agar tum ek dial ko thoda sa nudge karo, toh har bulb thoda aur bright ya dim ho jaata hai. Jacobian bas un saare "kitna bright per nudge" numbers ki table hai — har (bulb, dial) pair ke liye ek number. Agar tumhe yeh table pata hai, toh tum bina machine chalaye kisi bhi chhoti twist ka result predict kar sakte ho: apni twist ko bas table se multiply karo.
"ROC — Rows are Outputs, Columns are inputs." Aur: gradient Jacobian mein stack hota hai — har output ke gradient-rows stack karo.
f : R n → R m m × n
Jacobian ki ( i , j ) J ij = ∂ f i / ∂ x j
Jacobian ki row i Output f i ( ∇ f i ) ⊤
Scalar function (m = 1 Jacobian ek row vector hai jo ( ∇ f ) ⊤
Jacobian use karke first-order linear approximation likhо. f ( x + h ) ≈ f ( x ) + J h
h = g ∘ f J h = J g J f
∣ det J ∣ Map ka local volume-scaling factor; sirf tab defined hai jab J m = n
z = W x x W
Elementwise a = σ ( z ) z diag ( σ ′ ( z 1 ) , … , σ ′ ( z m ))
Polar map ( r cos θ , r sin θ ) det J r
Scalar derivative f prime x
Vector function f R^n to R^m
Partial derivatives dfi dxj
Column j is input xj response
Linear approximation requirement
Per-output first order expansion
Row times h equals grad fi dot h
ML backprop and optimization