Saliency maps and Grad-CAM: Worked examples — AI-ML deep dive

This page is the hands-on drill room for Saliency maps and Grad-CAM. The parent note built the ideas; here we grind through every case a saliency or Grad-CAM computation can throw at you — positive gradients, negative gradients, zero gradients, dead feature maps, degenerate images, and one exam-style trap. Nothing new is assumed: every symbol used below is re-earned here.

The scenario matrix

Every worked example below is tagged with the cell of this matrix it covers. Together they touch every cell.

Cell	What makes it different	Where it bites
A. Positive gradient	pixel/feature pushes score up	normal saliency pixel
B. Negative gradient	pixel pushes score down	why we take absolute value
C. Sign-mix across channels	R, G, B disagree	aggregation choice matters
D. Zero gradient (flat)	model insensitive here	dead / saturated region
E. Degenerate input	constant / blank image	gradient can vanish everywhere
F. Grad-CAM positive weight	feature map helps class	bright region
G. Grad-CAM negative weight	feature map hurts class	ReLU decision
H. Softmax-vs-logit trap	coupling across classes	the classic pitfall
I. Real-world word problem	"is the model cheating?"	debugging use-case
J. Exam twist	limiting / normalization edge	full-page all-zero map

Group 1 — Saliency at the pixel level

Worked example Example 1 — Cell A (positive gradient) & Cell B (negative gradient)

A tiny 1-channel network gives class score $S_{c} = 2 I_{1} + (- 3) I_{2} + 0 \cdot I_{3} .$ Here $I_{1}, I_{2}, I_{3}$ are three pixel values. Compute the saliency of each pixel.

Forecast: guess which pixel is most salient before reading on.

Differentiate w.r.t. each pixel. Why this step? Saliency is $∣ \partial S_{c} / \partial I ∣$ from the definition above — the sensitivity of the score to that pixel. For a linear score the derivative is just the coefficient. $\frac{\partial S _{c}}{\partial I _{1}} = 2, \frac{\partial S _{c}}{\partial I _{2}} = - 3, \frac{\partial S _{c}}{\partial I _{3}} = 0.$
Take absolute values. Why this step? Pixel $I_{2}$ has a negative gradient (Cell B): raising $I_{2}$ lowers the cat score. But "lowering the score a lot" is still influence — we care about magnitude, not direction. $Saliency = (∣2∣, ∣ - 3∣, ∣0∣) = (2, 3, 0) .$
Rank. Pixel $I_{2}$ is the most salient ( $3$ ), even though its raw gradient was negative.

Verify: Nudge $I_{2}$ up by $ϵ = 0.01$ . Score change $= - 3 (0.01) = - 0.03$ . Nudge $I_{1}$ up by the same: $+ 0.02$ . The bigger move ( $0.03$ ) came from $I_{2}$ — matches saliency ranking. ✓

The figure below plots exactly this example. The cyan bars are the raw signed gradients $(+ 2, - 3, 0)$ ; the amber bars are the saliencies $∣ \cdot ∣ = (2, 3, 0)$ . Follow the amber arrow: it points at pixel $I_{2}$ , whose bar is tallest after taking the absolute value even though its cyan (signed) bar dips below the white zero line. That single crossing is the whole lesson — sign lives on the cyan bars, importance lives on the amber ones.

Worked example Example 2 — Cell C (channel disagreement) & the max-vs-L2 choice

One RGB pixel has per-channel gradients $\frac{\partial S _{c}}{\partial ( R , G , B )} = (+ 4, - 3, 0) .$ Collapse to a single saliency value two ways.

Forecast: will "max" and "L2 norm" give the same number?

Max-abs aggregation. Why this step? We want one grey value per pixel to draw a heatmap. Max asks "what is the strongest single-channel effect?" $max (∣4∣, ∣ - 3∣, ∣0∣) = 4.$
L2-norm aggregation. Why this step? L2 asks "what is the combined magnitude if all channels move together?" $4^{2} + (- 3)^{2} + 0^{2} = 16 + 9 + 0 = 25 = 5.$
Compare. Max $= 4$ , L2 $= 5$ . They differ. L2 is $\geq$ max always, because it adds the other channels' energy on top.

Verify: L2 $\geq$ max because $\sum a_{i}^{2} \geq max a_{i}^{2} = max ∣ a_{i} ∣$ . Here $5 \geq 4$ . ✓ Both agree that channels R and G matter and B (=0) does not.

Worked example Example 3 — Cell D (zero gradient) & Cell E (degenerate blank image)

A ReLU unit outputs $S_{c} = max (0, w \cdot I + b)$ with $w = 1$ , $b = - 5$ , and a blank grey image where every pixel $I = 1$ , so the pre-activation is $1 - 5 = - 4$ .

Forecast: what saliency map do you expect for a blank image behind a "dead" ReLU?

Evaluate the pre-activation. Why this step? ReLU's gradient depends on whether its input is positive or negative. Input $= - 4 < 0$ .
Gradient of a switched-off ReLU. Why this step? For $max (0, z)$ with $z < 0$ , the output is flat at $0$ , so $\partial / \partial z = 0$ . By the chain rule the gradient to every pixel is $0 \cdot w = 0$ . $\frac{\partial S _{c}}{\partial I _{x, y}} = 0 everywhere.$
Saliency map. Every pixel $= ∣0∣ = 0$ : a completely black heatmap (Cell D + E together).

Verify: Perturb any pixel by $ϵ$ : pre-activation goes to $- 4 + ϵ$ , still negative for small $ϵ$ , ReLU still outputs $0$ , score unchanged. Zero saliency is correct, not a bug. ✓

This is the degenerate/limiting case: a saturated or dead unit gives you an all-zero map. If you see this in practice, the neuron isn't firing — don't trust the (empty) explanation.

Group 2 — Grad-CAM at the feature-map level

Worked example Example 4 — Cell F (positive weight) full Grad-CAM on a

$2 \times 2$ layer Two feature maps, each $2 \times 2$ : $A^{1} = [1001], A^{2} = [0220] .$ The gradients of the "dog" logit $S_{c}$ w.r.t. these maps are the constant grids $\frac{\partial S _{c}}{\partial A ^{1}} = [1111], \frac{\partial S _{c}}{\partial A ^{2}} = [0.5 0.5 0.5 0.5] .$

Forecast: which feature map dominates the heatmap?

Global-average-pool the gradients. Why this step? The weight $α_{k}^{c} = \frac{1}{Z} \sum_{i, j} \partial S_{c} / \partial A_{i, j}^{k}$ collapses each gradient grid to one importance number. Here $Z = u \cdot v = 4$ . $α_{1}^{c} = \frac{1}{4} (1 + 1 + 1 + 1) = 1, α_{2}^{c} = \frac{1}{4} (0.5 \cdot 4) = 0.5.$
Weighted sum of maps. Why this step? Each map is scaled by its importance, then added into one heatmap. $\sum_{k} α_{k}^{c} A^{k} = 1 \cdot [1001] + 0.5 \cdot [0220] = [1111] .$
ReLU. Why this step? Keep only positive evidence. All entries are $\geq 0$ , so ReLU changes nothing. $L = [1111] .$

Verify: $A^{1}$ ( $α_{1}^{c} = 1$ ) is weighted twice as strongly as $A^{2}$ ( $α_{2}^{c} = 0.5$ ), and the sum $1 \cdot A^{1} + 0.5 \cdot A^{2}$ evaluates cell-by-cell to $1$ everywhere. Uniform heatmap — the two maps' patterns exactly complement to fill the grid. ✓

The three panels below walk this computation left to right. The left two panels are the raw feature maps $A^{1}$ and $A^{2}$ with their pooled weights $α_{1}^{c} = 1$ and $α_{2}^{c} = 0.5$ printed in the titles; the right panel is the weighted sum $L$ , which comes out uniformly $1$ in every cell. Notice how $A^{1}$ 's bright diagonal and $A^{2}$ 's bright anti-diagonal interlock so that once each is scaled and added, no cell is left dark — that is why the final heatmap is flat.

Worked example Example 5 — Cell G (negative weight) & why ReLU is not optional

Same two maps as Example 4, but now the gradient for map 2 is negative: $\frac{\partial S _{c}}{\partial A ^{1}} = [1111], \frac{\partial S _{c}}{\partial A ^{2}} = [- 1 - 1 - 1 - 1] .$

Forecast: what happens to the top-right and bottom-left cells (where $A^{2}$ is large)?

Pool the gradients. Why this step? Exactly as in Example 4, we collapse each gradient grid to one weight $α_{k}^{c} = \frac{1}{Z} \sum_{i, j} \partial S_{c} / \partial A_{i, j}^{k}$ so that each feature map gets a single importance score before we combine them. $α_{1}^{c} = \frac{1}{4} (4) = 1, α_{2}^{c} = \frac{1}{4} (- 4) = - 1.$
Weighted sum (pre-ReLU). Why this step? See what the raw combination looks like before clipping. $1 \cdot [1001] + (- 1) \cdot [0220] = [1 - 2 - 2 1] .$
Apply ReLU. Why this step? Negative cells mean "this location argues against dog." For localization we only want positive evidence, so we clip them to $0$ . $L = ReLU [1 - 2 - 2 1] = [1001] .$

Verify: Without ReLU the $- 2$ cells would render as "strong" after we later take absolute value for display — falsely lighting up regions that hurt the class. ReLU leaves only the diagonal, exactly where $A^{1}$ (the helpful map) is active. ✓

This is the "Forgetting ReLU" mistake from the parent note, made concrete.

The two panels below are the before and after of Step 3. The left panel is the pre-ReLU sum, where the top-right and bottom-left cells sit at $- 2$ (rendered cool/blue). The right panel is post-ReLU, where those two negative cells have been clipped to $0$ , leaving only the warm diagonal at $+ 1$ . Reading left→right you literally watch ReLU erase the "arguments against dog" and keep the localization clean.

Group 3 — The softmax trap, a word problem, and an exam twist

Worked example Example 6 — Cell H (softmax vs logit coupling)

Two classes with logits $S_{cat} = 2$ , $S_{dog} = 1$ . Suppose a pixel change raises dog's logit by $δ$ but leaves cat's logit unchanged. Show that the softmax probability of cat still moves — even though $S_{cat}$ didn't.

Forecast: if $S_{cat}$ is untouched, can $p_{cat}$ really change?

Write softmax for cat. Why this step? This is the quantity people wrongly backprop through. $p_{cat} = \frac{e ^{S_{cat}}}{e ^{S_{cat}} + e ^{S_{dog}}} .$
Evaluate before. Why this step? We need the starting probability to measure any change against — this is the "control" reading before the pixel is touched. $p_{cat} = \frac{e ^{2}}{e ^{2} + e ^{1}} = \frac{7.389}{7.389 + 2.718} = 0.7311.$
Evaluate after, with dog raised by $δ = 0.5$ (so $S_{dog} = 1.5$ ), cat unchanged. Why this step? By recomputing with only the dog logit moved, we isolate the coupling effect: any change in $p_{cat}$ can only be blamed on dog, proving softmax leaks other classes into cat's explanation. $p_{cat} = \frac{e ^{2}}{e ^{2} + e ^{1.5}} = \frac{7.389}{7.389 + 4.482} = 0.6225.$
Interpret. $p_{cat}$ fell from $0.731$ to $0.622$ although $S_{cat}$ never moved. A saliency map built on $p_{cat}$ would blame this pixel for "hurting cat," which is a lie about cat features — it only helped dog.

Verify: raw-logit saliency uses $S_{cat}$ directly; its gradient here is $0$ (cat logit unchanged), which is the honest answer. Use logits, not softmax. ✓

Worked example Example 7 — Cell I (real-world "is the model cheating?" word problem)

A skin-cancer classifier scores $99%$ malignant on a photo. Grad-CAM on the $7 \times 7$ layer, after upsampling, puts all its mass in the bottom-left $2 \times 2$ corner where a ruler sticker sits, and none on the mole. Diagnose.

Forecast: is this a good model?

Read the heatmap semantics. Why this step? Bright Grad-CAM region = where positive evidence for "malignant" came from. Here that's the ruler, not the lesion.
Cross-check the dataset story. Why this step? Malignant training photos often include a ruler for scale; benign ones don't. The model learned "ruler ⇒ malignant" — a spurious shortcut.
Conclusion. The $99%$ is untrustworthy. This is exactly the debugging win Grad-CAM promises — see Model debugging.

Verify: hide the ruler (paint it out) and re-run; if the score collapses, the shortcut is confirmed. The heatmap predicted this before touching the pixels. ✓ (Compare with perturbation-based LIME and SHAP, which would reach the same verdict by masking regions.)

Worked example Example 8 — Cell J (exam twist: normalizing an all-negative heatmap)

Exam question: After ReLU your Grad-CAM raw map is $L = [0000]$ (every feature map had a negative $α_{k}^{c}$ ). You must normalize to $[0, 1]$ with $\hat{L} = \frac{L - min L}{max L - min L}$ . What goes wrong, and what should you report?

Forecast: what is $\frac{0 - 0}{0 - 0}$ ?

Compute min and max. $min L = 0$ , $max L = 0$ .
Plug into the normalizer. Why this step? To expose the trap. $\hat{L} = \frac{0 - 0}{0 - 0} = \frac{0}{0} (undefined) .$
Handle the degenerate limit. Why this step? Division by zero must be guarded. The honest report is "no positive evidence for class $c$ in this layer" — a flat, empty explanation — not a random pattern. Code should special-case $max = min$ and return the zero map.

Verify: every $α_{k}^{c} < 0$ means every feature map argues against the class; ReLU zeros the lot, so a blank map is the mathematically correct output. Reporting a normalized image here would fabricate structure that isn't there. ✓

Recall Self-test

Positive gradient $+ 2$ vs negative gradient $- 3$ : which pixel is more salient? ::: The $- 3$ pixel; saliency uses $∣ \cdot ∣$ , so $3 > 2$ . Max-abs vs L2 for $(+ 4, - 3, 0)$ ? ::: Max $= 4$ , L2 $= 5$ ; L2 is always $\geq$ max. Dead ReLU (input $- 4$ ) saliency? ::: Zero everywhere — a black map, and that is correct. Grad-CAM with $α_{2}^{c} = - 1$ on map $A^{2}$ : what does ReLU do? ::: Clips the negative cells to $0$ , keeping only positive-evidence regions. Why logits not softmax for saliency? ::: Softmax couples classes; another class's logit can move $p_{c}$ even when $S_{c}$ is fixed. All-negative $α_{k}^{c}$ → normalizing gives $0/0$ : report what? ::: A blank map — no positive evidence — never a fabricated pattern.

Prerequisites worth a look: Convolutional Neural Networks (feature maps), Attention mechanisms (a different "where is the model looking?" tool), and robustness contrasts in Adversarial examples.