Saliency maps and Grad-CAM — Hinglish notes

6.3.3 · AI-ML › Interpretability & Explainability

Saliency Maps Kya Hote Hain?

Scratch se derivation:

Hum kya chahte hain: Samajhna ki kaunse input features (pixels) ke liye model sensitive hai.
Neural networks kaise kaam karte hain: Output $S_{c}$ input $I$ ka ek differentiable function hai: $S_{c} = f (I; θ)$ .
Taylor expansion intuition: Agar hum pixel $I_{x, y}$ ko $ϵ$ se perturb karein, toh output approximately $\frac{\partial S _{c}}{\partial I _{x, y}} \cdot ϵ$ se change hoga. Bada gradient → bada impact.
Saliency score: Hum $\frac{\partial S _{c}}{\partial I _{x, y}}$ lete hain (absolute value isliye kyunki kisi pixel ko increase ya decrease karna dono score ko change kar sakte hain).
Computation: Network mein input layer ke respect mein ek backward pass (weights ke respect mein nahi).

Absolute value kyun?

Ek negative gradient matlab hai "is pixel ko decrease karna score ko increase karta hai," jo phir bhi important hai.
Hum influence ki magnitude ki care karte hain, direction ki nahi.

Saliency Compute Karna: Step-by-Step

Worked example Walkthrough: "Cat" Class ke liye Saliency Map

Setup: CNN image classify karta hai → logits output karta hai $[S_{cat}, S_{dog}, \dots]$ . Hum "cat" ke liye saliency chahte hain.

Step 1: Forward pass

Input image $I \in R^{H \times W \times 3}$ → Network Logits → $S_{cat}$ (abhi softmax nahi!)
Softmax kyun nahi? Softmax gradients ko squash kar deta hai. Hum raw class scores chahte hain. Step 2: Backward pass
$\nabla_{I} S_{cat}$ compute karo (cat-score ka gradient w.r.t. input pixels ke).
Shape: $R^{H \times W \times 3}$ (input jaisa hi).
Yeh step kyun? Chain rule humein $\frac{\partial S _{cat}}{\partial I}$ deta hai output se input tak backpropagating karke. Step 3: Color channels mein aggregate karo
RGB mein max lo: $Saliency (x, y) = max_{c} \frac{\partial S _{cat}}{\partial I _{x, y, c}}$
Ya L2 norm lo: $\sum_{c} (\frac{\partial S _{cat}}{\partial I _{x, y, c}})^{2}$
Max/norm kyun? Hum visualization ke liye har pixel ki ek value chahte hain.

Step 4: Visualize karo

[0, 1] pe normalize karo, colormap apply karo (hot = important, cold = irrelevant).

Concrete numbers:

Image: 224×224×3
Gradient: 224×224×3 (same shape)
Saliency map: 224×224 (grayscale heatmap)

Grad-CAM: Gradients ke Saath Class Activation Mapping

Yeh design kyun?

Derivation:

Observation: Deep conv layers high-level spatial information capture karte hain (jaise, "cat face yahan, paws wahan").
Goal: Har feature map ko weight karo "kitna yeh class $c$ mein contribute karta hai."
Gradient as importance: $\frac{\partial S ^{c}}{\partial A ^{k}}$ class score ki sensitivity measure karta hai feature map $k$ ke respect mein.
Global average pooling: Pixel-wise weights ki jagah, spatial locations mein gradients average karo: $α_{k}^{c} = \frac{1}{u \cdot v} \sum_{i, j} \frac{\partial S ^{c}}{\partial A _{i, j}^{k}}$ . Yeh har feature map ke liye ek weight deta hai.
- Average kyun? Feature maps spatially coarse hote hain; hum us feature ki overall importance chahte hain.
Linear combination: $\sum_{k} α_{k}^{c} A^{k}$ har feature map ko weight karta hai, ek single heatmap mein sum karke.
ReLU: Negative values "kya score decrease karta hai" highlight karti hain (localization ke liye kam interpretable).
Upsample: Feature map input se chhota hota hai (jaise, 7×7 vs. 224×224). Image size tak bilinear interpolation.

Worked example Grad-CAM: 224×224 Image mein "Dog" Classify Karna

Architecture: ResNet-50, last conv layer = 7×7×2048 (2048 feature maps, 7×7 spatial). Step 1: Forward pass → $S_{dog}$ (dog class ke liye logit). Step 2: $\frac{\partial S _{dog}}{\partial A}$ compute karo (gradient w.r.t. last conv layer, shape 7×7×2048).

Step 3: Spatial dimensions mein global average pooling: $α_{k}^{dog} = \frac{1}{49} \sum_{i = 1}^{7} \sum_{j = 1}^{7} \frac{\partial S _{dog}}{\partial A _{i, j}^{k}}$ Ab hamare paas 2048 scalars hain (har feature map ke liye ek). Step 4: Feature maps ka weighted sum: $L = \sum_{k = 1}^{2048} α_{k}^{dog} \cdot A^{k}$ Result: 7×7 heatmap.

Step 5: ReLU (negative values zero karo) → 224×224 tak upsample karo → Image pe overlay karo.

Interpretation: Bright regions = model yahan "dog" classification ke liye focused tha. Agar yeh dog ke face ko light up karta hai: achha. Agar yeh watermark ko light up karta hai: model cheat kar raha hai.

Saliency vs. Grad-CAM: Key Differences

Aspect	Saliency Maps	Grad-CAM
Granularity	Pixel-level (high-res)	Region-level (coarse, phir upsample)
Layer	Input layer gradients	Last conv layer activations + gradients
Kya dikhata hai	Kaunse pixels score change karte hain	Kaunse spatial regions class $c$ ke liye activate hote hain
Noise	Bahut noisy (pixel gradients chaotic hote hain)	Smoother (feature maps info aggregate karte hain)
Use case	Fine-grained debugging	Localization, sanity checks

Saliency ke upar Grad-CAM kyun?

Deep layers mein feature maps objects/parts encode karte hain (edges → textures → parts → objects). Saliency maps pixel-level aur noisy hote hain.
Grad-CAM class-discriminative hai: tum visualize kar sakte ho "model ne cat kahan dekhi vs. dog kahan dekha?"

Common mistake Common Pitfall: Gradients Se Pehle Softmax Use Karna

Galat approach:

logits = model(image)
probs = softmax(logits)
loss = probs[class_idx]
loss.backward()  # gradient w.r.t. input

Kyun sahi lagta hai: "Main jaanna chahta hoon ki probability ko kya influence karta hai."

Kyun galat hai: Softmax saari classes ko couple karta hai. $p_{c} = \frac{e ^{S_{c}}}{\sum _{k} e ^{S_{k}}}$ ka gradient w.r.t. input mein sabhi logits involve hote hain. Agar koi doosra class score increase hota hai, $p_{c}$ decrease hota hai chahe $S_{c}$ unchanged ho. Tumhe noisy, confusing saliency milti hai.

Fix: Raw logits use karo: loss = logits[class_idx]. Yeh class score ki sensitivity ko isolate karta hai.

Common mistake Grad-CAM mein ReLU Bhoolna

Galat:

cam = (alpha * feature_maps).sum(axis=0)  # No ReLU

Kyun sahi lagta hai: "Negative weights positives ko cancel kar sakte hain, poori picture dikhate hue."

Kyun galat hai: Negative regions class score ko suppress karte hain (jaise, "background dog confidence decrease karta hai"). Localization ke liye, hum sirf care karte hain jahan model ne class ke liye positive evidence paaya. Fix:

cam = relu((alpha * feature_maps).sum(axis=0))

Yeh negative contributions zero kar deta hai, sirf class-specific positive regions rakhte hue.

Recall Feynman: Ek 12-Saal Ke Bacche Ko Explain Karo

Socho tumne ek robot train kiya photos mein cats recognize karne ke liye. Tum use ek picture dikhate ho, aur woh kehta hai "90% sure yeh cat hai!" Lekin usne decide kaise kiya?

Saliency maps: Hum robot se poochte hain, "Agar main yeh ek pixel thoda sa change karun, toh tumhari 'cat confidence' kitni change hogi?" Agar cat ke kaan mein ek pixel change karna jawab bahut change karta hai, toh woh pixel important hai. Hum important pixels ko bright red aur unimportant waalon ko dark blue color karte hain. Ab hum dekhte hain ki robot kaan aur whiskers dekh raha tha!

Grad-CAM: Har ek pixel ke baare mein poochhne ki jagah (jo messy ho jaata hai), hum robot ke andar ki "brain layers" se deeper poochte hain. Final answer se pehle ki last layer mein mini-images hoti hain jo dikhati hain "yahan fur hai, wahan pointy ears hain." Hum poochte hain, "Inme se kaunsi mini-images ne tumhe 'cat' sochne par majboor kiya?" aur un regions ko color karte hain. Phir hum us colored map ko blow up karte hain aur original photo ke upar rakh dete hain. Ab hum dekhte hain ki robot cat ke face pe focused tha, peeche ke couch pe nahi.

Yeh kyun important hai: Agar robot sirf background dekhta hai (jaise hamesha cats ko blue carpets pe dekhna), toh woh red carpet pe cat pe fail kar jaayega. Yeh maps humein robot ko "cheat karte" pakadne mein help karte hain.

Connections

Backpropagation: Saliency/Grad-CAM standard backprop ke gradients use karte hain, bas weights ki jagah input/features ko target karke.
Convolutional Neural Networks: Grad-CAM conv layers ki spatial structure leverage karta hai; dense layers ke liye kaam nahi karta.
Model debugging: Saliency/Grad-CAM spurious correlations reveal karte hain (Clever Hans effects).
Attention mechanisms: Attention weights dikhate hain "model har step par kya attend karta hai"; Grad-CAM CNNs mein spatial attention dikhata hai.
Adversarial examples: High-saliency regions adversarial perturbations ke liye vulnerable hote hain.
LIME and SHAP: Gradient-based saliency ke model-agnostic alternatives; Grad-CAM gradient-specific hai lekin faster hai.

#flashcards/ai-ml

Saliency map kya hota hai? :: Ek visualization jo output ke gradient ki magnitude dikhata hai har input pixel ke respect mein: $∣ \frac{\partial S _{c}}{\partial I _{x, y}} ∣$ . High values un pixels ko indicate karte hain jo prediction ko strongly influence karte hain.

Saliency ke liye hum softmax probabilities ki jagah raw logits kyun use karte hain?

Softmax saari classes ko couple karta hai—ek class score change karna saari probabilities affect karta hai. Raw logits target class ki sensitivity ko input changes ke respect mein isolate karte hain, cleaner gradients dete hue.

Grad-CAM ka full form kya hai?

Gradient-weighted Class Activation Mapping. Yeh last convolutional layer mein flow hone wale gradients ko feature maps weight karne ke liye use karta hai, ek class-specific localization heatmap produce karke.

Grad-CAM formula likhо :: $L_{Grad-CAM}^{c} = ReLU (\sum_{k} α_{k}^{c} A^{k})$ , jahan $α_{k}^{c} = \frac{1}{Z} \sum_{i, j} \frac{\partial S ^{c}}{\partial A _{i, j}^{k}}$ (global average pooled gradients) aur $A^{k}$ $k$ -th feature map hai.

Grad-CAM final heatmap pe ReLU kyun apply karta hai?

ReLU negative values zero kar deta hai, sirf un regions ko rakhte hue jo target class score mein positively contribute karte hain. Negative regions class ko suppress karte hain (localization ke liye kam interpretable).

Saliency maps aur Grad-CAM mein key difference kya hai? :: Saliency maps pixel-level gradients hote hain (noisy, high-res). Grad-CAM last conv layer feature maps use karta hai (coarse, semantic, class-discriminative).

Grad-CAM mein, hum last convolutional layer kyun use karte hain?

Deep conv layers high-level spatial features (object parts, semantic regions) capture karte hain spatial structure retain karte hue. Earlier layers bahut low-level hoti hain; baad ki dense layers spatial info kho deti hain.

Grad-CAM mein

α_{k}^{c}

kaise compute karte hain?

Gradients ka global average pooling:

α_{k}^{c} = \frac{1}{u \cdot v} \sum_{i, j} \frac{\partial S ^{c}}{\partial A _{i, j}^{k}}

. Yeh har feature map ke liye ek importance weight deta hai.

Grad-CAM heatmap ek misclassification ke baare mein kya batata hai?

Yeh dikhata hai ki model ne (galat) class ke liye kaunse spatial regions use kiye. Agar yeh irrelevant features (background, watermarks) highlight karta hai, toh model ne spurious correlations seekhi hain.

Grad-CAM class-discriminative kyun hai?

Weights

α_{k}^{c}

class

c

ke score ke respect mein gradient pe depend karte hain, isliye alag-alag classes alag heatmaps produce karti hain jo dikhate hain ki model ne har class ke liye evidence kahan paaya.