Saliency maps and Grad-CAM: Foundations — AI-ML deep dive

Before you can read the parent note 6.3.3 Saliency maps and Grad-CAM, you need to own every symbol it throws at you. This page builds each one from zero, in an order where nothing appears before it is defined. Nothing here is memorised — everything is earned.

0. The picture we keep coming back to

Everything in this topic lives on one mental image: an image goes in on the left, a number comes out on the right, and there is a chain of computations in between. Keep this picture in your head — every symbol below is a label on some part of it.

Look at the figure: the grid on the left is the image, the box in the middle is the network, and the single glowing dot on the right is the score. We will now name every piece.

1. What is an image, really? — the symbol $I$

Plain words: a photo is a spreadsheet of brightness values.
The picture: the left grid in Figure 1 — each cell holds one number.
Why the topic needs it: saliency asks "how important is each pixel?" You cannot talk about a pixel's importance until "pixel" is a numbered thing you can point to.

Colour images have three such grids stacked: one for Red, one for Green, one for Blue. We index the colour with a letter $c_{ch}$ (a channel index, not to be confused with the class label $c$ ), so a full colour pixel is $I_{x, y, c_{ch}}$ .

For a $224 \times 224$ colour image that block holds $224 \times 224 \times 3 = 150, 528$ numbers.

2. The network as a function — $f$ , $θ$ , and $S_{c}$

Plain words: $f$ is the whole recipe that turns pixels into a cat-score.
The picture: the middle box → the right dot in Figure 1.
Why "function"? Because a function is exactly a machine with one output per input — and being a function is what lets us do calculus on it later.

3. The building block: slope of one variable — the derivative

Before gradients (many inputs) we need the derivative (one input).

Plain words: it's the steepness of the score-vs-pixel curve at your current pixel value.
The picture: Figure 2 — the red tangent line's tilt is the derivative. Steep line ⇒ big derivative ⇒ this pixel matters a lot.
Why the topic needs it: this is literally the definition of saliency. Saliency of a pixel = the size of this steepness.

4. From one slope to many — the gradient $\nabla_{I} S_{c}$

Plain words: one derivative per pixel, laid back out in image shape.
The picture: a full grid where each cell holds its own steepness number — see Figure 3.
Why the topic needs it: the raw saliency map, before any cleanup, is exactly $\nabla_{I} S_{c}$ . Shape $R^{H \times W \times 3}$ , same as the image.

5. How the gradient is computed — forward & backward pass

Why w.r.t. the input, not the weights? Training computes gradients w.r.t. $θ$ (to improve the model). Here $θ$ is frozen; we redirect the same machinery to compute the gradient w.r.t. $I$ instead. Same tool, different target.

6. Collapsing colour — max and L2 norm

The gradient has 3 numbers per pixel (one per RGB channel). A heatmap needs one number per pixel.

7. Feature maps $A^{k}$ — what Grad-CAM works on

Saliency reads the input. Grad-CAM reads deep inside the network, at the last convolutional layer (see Convolutional Neural Networks).

Plain words: deep layers no longer store pixels; they store "how strongly pattern $k$ fires at each coarse location."
The picture: Figure 4 — a tall stack of small $u \times v$ grids ( $7 \times 7$ in ResNet-50), $k$ running over the 2048 grids.
Why the topic needs it: Grad-CAM weights these by importance instead of raw pixels — that is why its heatmaps are smooth, not noisy.

8. The Grad-CAM weights $α_{k}^{c}$ — averaging gradients

9. ReLU — keeping only the positive evidence

10. Upsampling — from $7 \times 7$ back to $224 \times 224$

Prerequisite map — and how to read it

The flowchart below is the table of contents of this page as a dependency graph: each box is a section you just read, and each arrow means "you must understand the box behind the arrow before the box in front makes sense." Trace it once from top to bottom to see why we introduced the symbols in this exact order.

Reading it in words: the image (top) feeds the network; the network gives a score; the score gives a derivative, which fans out into the full gradient. From the gradient, one path builds the pixel-level saliency map, and another path (via the deep feature maps and their averaged gradients) builds the coarse Grad-CAM map, which is upsampled into the final heatmap. Both paths converge on the same goal: interpretability and debugging. Notice the graph has exactly the same shape as the sections you just walked through — nothing new is hidden in it.

This same gradient toolkit underpins neighbours like Adversarial examples (nudging pixels to fool the model) and complements model-agnostic tools in LIME and SHAP. Use all of them for Model debugging; contrast the "where it looks" idea with Attention mechanisms.

Equipment checklist

Test yourself — reveal only after answering aloud.

What does $I_{x, y, c_{ch}}$ mean? ::: The brightness number of the pixel at column $x$ , row $y$ , colour channel $c_{ch}$ (R, G, or B).
What is $θ$ and do we change it in saliency? ::: The network's weights; no — they are frozen throughout.
In plain words, what is $S_{c}$ ? ::: The raw score (logit) for class $c$ — bigger means more confident. (We use this one symbol everywhere.)
Which class $c$ should you pick, and when? ::: The predicted class to explain the model's own decision; a named target class to ask "where is the evidence for that class specifically."
What real-world quantity does $\frac{\partial S _{c}}{\partial I _{x, y}}$ measure? ::: How much the class score changes per tiny change in that one pixel — the pixel's influence.
Why take the absolute value in a saliency map? ::: A negative gradient still means the pixel is influential; we want magnitude, not direction.
What does the nabla $\nabla_{I} S_{c}$ collect? ::: One derivative per pixel, arranged back into image shape (the full gradient).
State the chain rule for $S_{c} = q (g (I))$ . ::: $\frac{\partial S _{c}}{\partial I} = \frac{\partial S _{c}}{\partial h} \cdot \frac{\partial h}{\partial I}$ with $h = g (I)$ — multiply the little slopes.
Why must the network be differentiable? ::: Only a smooth function lets us ask "how much does the output wiggle when the input wiggles."
What are $u$ , $v$ , and $Z$ ? ::: $u$ = feature-map rows, $v$ = feature-map columns, $Z = u \cdot v$ = cells per map (49 for 7×7).
What is $A^{k}$ ? ::: The $k$ -th feature map (pattern-detector grid) of the last convolutional layer.
What does $α_{k}^{c}$ represent and how is it built? ::: The overall importance of feature map $k$ for class $c$ , made by averaging that map's gradients over all its $u \times v$ cells.
What does $ReLU (x)$ do and why use it in Grad-CAM? ::: Keeps positives, zeros negatives; keeps only regions that raise the class score for localization.
After ReLU, what final step makes Grad-CAM overlay on the image, and how? ::: Upsampling by bilinear interpolation from $u \times v$ up to $H \times W$ .