Adversarial examples and robustness: Visual walkthrough

Everything below assumes only that you know a neural network takes an input and outputs a guess. We build the rest. If you have never met a trained network or gradients, this page still starts from line one.

Step 1 — What is "the input" really? A point in a huge space

WHAT. Before any attack, we need to picture the thing we are attacking: the input image.

Think of a tiny image with just two pixels. Pixel one has a brightness value; pixel two has a brightness value. So the whole image is just two numbers, like $(0.4, 0.7)$ . Two numbers = one point on a flat plane, with the horizontal axis being "pixel 1 brightness" and the vertical axis being "pixel 2 brightness."

WHY. Real images have $150, 528$ numbers (a $224 \times 224$ colour image), which we cannot draw. But the idea is identical in 2 dimensions and in 150 thousand — a point in a space with one axis per pixel. So we draw the 2-pixel version and trust the picture scales up.

PICTURE. The orange dot is our real image $x$ . It sits somewhere on the "cat" side of a curved line.

Figure — Adversarial examples and robustness

Step 2 — What is "the loss"? A hill over that space

WHAT. The network doesn't just say "cat" or "dog." It produces a number, the loss $L$ , that measures how wrong it currently is about the true answer. Small loss = confident and correct. Large loss = confused or wrong.

WHY. We need a single number we can push up. "Fool the model" is vague; "make the loss go up" is a precise target. Every attack in this field is secretly the sentence: raise the loss while barely moving the pixels.

PICTURE. Imagine a landscape floating above the pixel-plane. At each point $x$ the height of the land is the loss $L$ . Our orange dot sits in a valley (low loss — the model is happy and correct). Nearby, over the ridge, the land rises into wrongness.

Step 3 — Which way is uphill? The gradient arrow

WHAT. Standing on the loss-hill at our orange dot, we ask: which direction makes the land rise fastest? That direction is the gradient, written $\nabla_{x} L$ .

WHY THIS TOOL. We want to increase loss with the smallest possible move. Out of all directions you could step, the gradient is provably the single steepest-uphill one. No other tool answers "steepest ascent" — that is exactly what a gradient is. (The same object that gradient descent follows downhill to train; we simply read it as pointing uphill instead.)

PICTURE. The purple arrow leaves the orange dot pointing straight up the slope, perpendicular to the loss contour lines (the "height rings" of the hill). Its two components tell us how steep the hill is along pixel 1 and along pixel 2 separately.

Step 4 — Pretend the hill is a flat ramp (the linear approximation)

WHAT. Instead of the real curvy hill, we replace it, just near our dot, with a straight tilted ramp. A tiny step $δ$ then changes the loss by a clean, predictable amount.

WHY. The true hill is complicated. But we only plan a tiny move (imperceptible perturbation). Zoom into any smooth hill far enough and it looks flat — like the Earth looks flat from your street. On a flat ramp the maths is trivial, and "trivial maths near our point" is exactly the first-order Taylor expansion. This is why we use it and not the full loss.

PICTURE. The curved contour hill of Step 3, with a translucent flat plane resting tangent to it at the orange dot. Over the small circle of allowed moves, plane and hill barely differ.

Step 5 — The dot product: how much a step "agrees" with uphill

WHAT. The gained loss is $δ^{⊤} \nabla_{x} L$ . This operation — multiply matching components and add them up — is the dot product. It measures how much two arrows point the same way.

WHY THIS TOOL. Our goal is "gain the most loss." The dot product is precisely the tool that scores agreement of direction. It is biggest when our step $δ$ aims along the gradient, zero when the step is sideways (perpendicular), and negative when we walk downhill. No other quantity captures "how aligned are these two arrows" so directly.

PICTURE. Three copies of the same gradient arrow. A step aligned with it → tall green loss-gain bar. A sideways step → zero bar. An opposed step → negative (red) bar.

So the whole attack now reads: choose $δ$ to make this sum as large as possible.

Step 6 — The budget: how big is a tiny step allowed to be?

WHAT. We cannot just crank $δ$ to infinity — the change must stay invisible. We cap the biggest single-pixel move at a small number $ϵ$ . That cap is the $L_{\infty}$ norm constraint.

WHY. "Invisible" needs a definition. The $L_{\infty}$ norm reads off the largest pixel change and says "keep even the worst one under $ϵ$ ." If no single pixel moves more than $ϵ = 8/255$ , the human eye sees nothing. That is why this norm rules the field: it maps straight onto "no pixel changed noticeably."

PICTURE. In our 2-pixel plane the set of legal $δ$ is a square box of half-width $ϵ$ centred on the dot. (A circle would be the $L_{2}$ norm; a square is $L_{\infty}$ , because "each coordinate ≤ $ϵ$ " is a box.) The orange dot sits at the box's centre.

Step 7 — The punchline: aim every pixel with `sign`

WHAT. Maximise the dot product $\sum_{i} δ_{i} \cdot \frac{\partial L}{\partial x _{i}}$ while every $δ_{i}$ is trapped in $[- ϵ, + ϵ]$ . Solve it pixel by pixel.

WHY. Look at one term $δ_{i} \cdot \frac{\partial L}{\partial x _{i}}$ . If the slope $\frac{\partial L}{\partial x _{i}}$ is positive, the term grows fastest by setting $δ_{i} = + ϵ$ (push up as far as allowed). If the slope is negative, set $δ_{i} = - ϵ$ (push down). Either way the best move is $ϵ$ times the sign of the slope. Because each pixel is independent inside a box, solving each one separately is exactly optimal — that is the whole trick.

PICTURE. The square box from Step 6. The gradient arrow points into some quadrant. The best legal $δ$ is not along the arrow — it jumps to the corner of the box that the arrow leans toward. Each coordinate slams to $+ ϵ$ or $- ϵ$ .

Step 8 — Edge and degenerate cases (never skip these)

WHAT & WHY & PICTURE, all in one panel — the situations the clean formula quietly assumes away:

Zero gradient ( $\frac{\partial L}{\partial x _{i}} = 0$ ). $sign (0) = 0$ , so that pixel is not moved. The hill is flat there — nudging it gains nothing. Correct behaviour.
Already at the boundary. If our dot sits right on the cat/dog line, even a sub- $ϵ$ step flips the label — FGSM overkills easily. Robust models push this boundary far from data.
$ϵ$ too large. The flat-ramp promise of Step 4 dies: we walk off the ramp onto real curvature, the linear estimate misfires, and the attack can miss. This is exactly the gap PGD fills — take small steps $α$ , re-read the gradient, re-aim, and stay inside the box by clipping.
Targeted attack. To force a chosen wrong class $t$ , flip the sign: $x_{adv} = x - ϵ \cdot sign (\nabla_{x} L (θ, x, t))$ . Minus means "walk downhill on class $t$ 's loss" — make the model confident in $t$ .
Curse of dimensionality. With $d$ pixels all pushed by $ϵ$ and all agreeing in sign, total loss gain scales like $ϵ d$ — enormous when $d = 150, 528$ , even though each pixel barely moved. This is why the attack is so devastating in high dimensions.

The one-picture summary

The whole story in one frame: the orange dot sits in the cat valley; the purple gradient reads off steepest-uphill; the square box caps how far we may move; the attack jumps to the box corner ( $sign$ ); one dot's-throw later we cross the boundary and the model shouts "dog."

Recall Feynman retelling — say it like a story

Imagine a landscape where low ground means the AI is right and high ground means it's wrong. Your photo of a cat sits in a comfortable valley. Now, without letting anyone see you nudge the photo, you want to walk it uphill into wrongness — but every pixel is only allowed to shift by a hair, $ϵ$ .

First you feel the ground: which way is uphill? That's the gradient — one slope reading per pixel. Then you realise you don't have to follow that arrow exactly; you're stuck inside a tiny square fence of allowed moves. The highest point you can touch inside a square is always a corner, and to reach a corner you shove every pixel to its limit — up if that pixel's slope is uphill, down if it's downhill. That "up-or-down, full tilt" decision is the sign. Do it once, for all pixels at once, and you've built the adversarial image.

Why does such a whisper work? Because there are 150 000 pixels, and 150 000 tiny agreeing shoves add up to a big climb. A picture that still looks exactly like a cat to you now sits high on the wrong-hill, and the network confidently calls it a dog. That's FGSM.

Recall Quick self-test

Why do we use sign instead of the raw gradient? ::: Because the budget is an $L_{\infty}$ box, and the maximum loss inside a box is at a corner; only pushing each coordinate to $\pm ϵ$ (i.e. $ϵ \cdot sign$ ) reaches a corner. What does a zero gradient component do to that pixel? ::: Nothing — $sign (0) = 0$ , the pixel stays put because the hill is flat there. Why subtract for a targeted attack? ::: To lower the target class's loss, i.e. walk downhill on class $t$ , making the model confident in $t$ . Why does the attack get stronger in higher dimensions? ::: Loss gain scales like $ϵ d$ ; more pixels each contributing $ϵ$ of agreeing push sums to a large climb.

See also: AI Safety fundamentals · Out-of-distribution detection · Interpretability and explainability · Reward hacking · Gradient descent optimization · Neural network training