VGG networks: Visual walkthrough — AI-ML deep dive

Before line one, three words we will use constantly:

Step 1 — One 3×3 filter sees a 3×3 window

WHAT. Place a 3×3 filter on the input. It covers a square of 9 pixels and produces a single output number.

WHY. We must first agree on the smallest possible "seeing". Everything later is built by stacking this one move, so we nail it down first.

PICTURE. (Figure s01: a horizontal strip of white input-pixel boxes; a 3-wide amber block marks the pixels one filter touches; an arrow rises from it to a single cyan output dot labelled "output number".) The amber square marks the 9 input pixels the filter touches. The cyan dot is the single output number they collapse into. That amber square is the receptive field of that one cyan dot.

$RF_{1} = 3$

$RF_{1}$ ::: the width (in input pixels) of the window seen after layer 1 (the first convolution).
The number $3$ ::: the filter is 3 pixels wide, so one layer sees a 3-wide window. Nothing mysterious yet.

Step 2 — Stack a second 3×3 on top

WHAT. Feed the output of layer 1 into a second 3×3 filter. This second filter looks at a 3×3 window — but now those are layer-1 outputs, not original pixels.

WHY. VGG never uses a big filter. Its only tool for "seeing wider" is to add another small filter on top. So we ask: after two 3×3 layers, how many original pixels does the final number depend on?

PICTURE. (Figure s02: the same input strip at the bottom; three cyan "layer-1" dots, each reaching down over its own 3-wide box; one amber "layer-2" dot at the top reading all three; a double-headed bracket underneath spanning original pixels 1–5.) The top amber dot (layer 2) reads three neighbouring layer-1 dots. Trace each of those three down: each one already reached down into its own 3-wide strip of original pixels. The three strips overlap but together span pixels $1$ through $5$ .

Count it directly from the picture: the three layer-1 dots cover original columns ${1, 2, 3}$ , ${2, 3, 4}$ , ${3, 4, 5}$ . Their union is ${1, 2, 3, 4, 5}$ — five pixels.

$RF_{2} = 5$

$RF_{2}$ ::: width of original-pixel window seen after layer 2 (two stacked convolutions).
The jump $3 \to 5$ ::: each new 3×3 layer reaches 1 extra pixel on each side, so the window grows by 2.

Step 3 — Turn the picture into a formula

WHAT. Write the pattern "each layer adds its reach" as one reusable equation, indexed by the layer number $n$ .

WHY. So far we counted by hand. That works for width 5, but VGG stacks three filters and pools with strides. We need a rule that survives all those cases without re-drawing every time. Since the rule changes layer by layer, we tag each quantity with $n$ = "which layer".

PICTURE. (Figure s03: the amber bracket = "reach we already had", plus two small cyan boxes = "+1 pixel each side the new layer adds"; below them the formula written out.) The figure shows the reach we already had (amber bracket) and the extra reach added by the new layer (cyan boxes, one pixel per side). The total bracket is their sum — that visual sum is the formula.

$RF_{n} = RF_{n - 1} + (k_{n} - 1) \times \prod_{i = 1}^{n - 1} s_{i}$

Reading it symbol by symbol, exactly where each sits (recall $n$ = layer number from the definition box):

$RF_{n}$ ::: the receptive-field width after layer $n$ — what we're solving for.
$RF_{n - 1}$ ::: the window we already had before adding layer $n$ — the amber bracket.
$k_{n}$ ::: the width of the filter at layer $n$ . For VGG, $k_{n} = 3$ everywhere.
$(k_{n} - 1)$ ::: the extra reach layer $n$ adds. For a 3-wide filter, $3 - 1 = 2$ — the "+2" we saw.
$s_{i}$ ::: the stride of an earlier layer $i$ — how many pixels that filter jumped each step.
$\prod_{i = 1}^{n - 1} s_{i}$ ::: multiply together all strides from layer 1 up to layer $n - 1$ .

Check against Steps 1–2 (strides all 1): $RF_{1} = 3, RF_{2} = 3 + (3 - 1) \times 1 = 5. ✓$

Step 4 — Three 3×3s equal one 7×7

WHAT. Add a third 3×3 filter and compute the receptive field again.

WHY. VGG's deeper blocks stack three 3×3 convs. We want to know the big filter they secretly imitate.

PICTURE. (Figure s04: three amber double-headed brackets telescoping outward, labelled RF=3, RF=5, RF=7; the outermost spans a 7-wide chunk.) Three windows telescope outward: $3 \to 5 \to 7$ . The outermost bracket spans a 7-wide chunk of the original image — the same span a single 7×7 filter would grab. (And by the square-symmetry note, that means a full 7×7 in 2-D.)

$RF_{3} = RF_{2} + (3 - 1) \times 1 = 5 + 2 = 7$

$RF_{3} = 7$ ::: after three stacked 3×3 layers, each output sees a 7×7 original window.

Step 5 — Count the weights: small stack wins

WHAT. Count the learnable numbers (weights) for the big filter versus the small stack, at equal input/output channel width $C$ .

WHY. Fewer weights → less overfitting, faster training, less memory. This is the payoff that made VGG famous. We ignore biases here to compare the pure filter cost.

PICTURE. (Figure s05: a tall amber bar labelled "one 7×7 = 49 C²" next to three short cyan bars stacked to "27 C²", with a "45% fewer weights" note between them.) The tall amber bar is the single 7×7's weight count; three short cyan bars (each $9 C^{2}$ ) stack up shorter than the amber bar.

One conv layer from $C$ input channels to $C$ output channels, filter $k \times k$ , costs: $P = k \times k \times C \times C = k^{2} C^{2}$

$k \times k$ ::: numbers in one 2-D filter.
first $C$ ::: each filter reads all $C$ input channels, so it's really $k \times k \times C$ .
second $C$ ::: we build $C$ such filters, one per output channel.

Now compare:

$one 7 \times 7 : 49 C^{2} three 3 \times 3 : 3 \times 9 C^{2} = 27 C^{2}$

$saving = \frac{49 - 27}{49} \approx 45%$

And the two-layer case from the parent (two 3×3 vs one 5×5): $two 3 \times 3 = 18 C^{2} < 25 C^{2} = one 5 \times 5, saving = \frac{25 - 18}{25} = 28%.$

Step 6 — The degenerate cases (don't skip these)

WHAT. Check the edges: a $1 \times 1$ filter, and stride $> 1$ (pooling).

WHY. The contract says cover every case. If our "+2 per layer" rule breaks somewhere, we must know where.

PICTURE. (Figure s06, left panel: a $1 \times 1$ filter drawn as a single amber box with an arrow straight up to one cyan dot — zero reach beyond itself. Right panel: a stride-2 step drawn with widely-spaced dots and a bracket spanning 4 original pixels.) Left: a $1 \times 1$ filter is a single dot, zero reach beyond itself. Right: a stride-2 pooling step doubling how far the next layer's reach spreads.

Case A — the $1 \times 1$ filter ( $k = 1$ ). $(k - 1) = (1 - 1) = 0$ It adds zero spatial reach — it mixes channels but sees no neighbours. This is exactly why the parent warns that canonical VGG-16/19 use only 3×3: a $1 \times 1$ can't grow the receptive field, so it can't replace spatial filters. (It appears only in the experimental "config C".)

Case B — stride $s = 2$ (max-pool). After a pooling layer, the product $\prod s_{i}$ picks up a factor of 2. So the next conv's reach is doubled in original-pixel terms: $extra reach = (3 - 1) \times 2 = 4 original pixels .$ This is why VGG's receptive field explodes deep in the network even though every filter stays 3×3 — pooling multiplies the reach (exactly the stride-product effect from Step 3). It's also why VGG doubles channels after each pool (parent Step "Why Double Channels"): halving height and width quarters the activation count, so widening $C \to 2 C$ keeps representational capacity from collapsing.

The one-picture summary

(Figure s07: on top, three telescoping brackets 3→5→7 with tiny "ReLU" tags between filters; on the bottom, the amber 49C² bar beside the three-stacked cyan 27C² bars; a caption "same view, fewer weights, more ReLUs".) The whole story on one drawing: three tiny 3×3 filters telescope out to a 7×7 view, while their weight bars stay shorter than the single 7×7 bar, with a ReLU bend tucked between each pair.

Recall Feynman retelling — say it back in plain words

A filter is a little sliding grabber, and we number the layers $1, 2, 3$ in the order the picture flows through them. One 3×3 grabber sees a 3-pixel-wide patch. Put a second grabber on top and it sees 5 pixels of the original; a third sees 7 — and because a 3×3 is square, "7 wide" really means a full 7×7 square. So three cheap little grabbers see just as wide as one fat 7×7 grabber — but the fat one carries $49 C^{2}$ numbers while the three thin ones carry only $27 C^{2}$ , about 45% less to learn and to overfit. Better still, we get to bend the signal (ReLU) between every grabber, so the stack draws curvier decision boundaries than the single big filter ever could. Padding doesn't touch any of this reach counting — it only keeps the feature-map grid the same size (that's "same" padding). Two special cases: a $1 \times 1$ grabber adds no width (it only mixes channels), and a stride-2 pool doubles how far the next grabber reaches — because earlier strides multiply distance in the original image, which is exactly the product term in our formula. That doubling is the engine that makes deep VGG's view grow huge, and it's why we double the channel count after every pool to keep the network's memory from thinning out.

Recall Self-test

Receptive field after four stacked 3×3 conv layers, all stride 1? ::: $3 + 2 + 2 + 2 = 9$ . Weights (biases ignored) for two 3×3 layers, $C \to C$ each? ::: $18 C^{2}$ . Why does a $1 \times 1$ filter fail to widen the receptive field? ::: because $(k - 1) = 0$ , so it adds zero spatial reach. After a stride-2 pool, how much does the next 3×3's original-pixel reach grow? ::: $(3 - 1) \times 2 = 4$ pixels. What does the little index $n$ count in $RF_{n}$ ? ::: the layer number — which convolution the picture has passed through. Does padding change the receptive field? ::: no — it only changes the feature-map (grid) size.

Related: VGG networks · 3.4.05-alexnet (the big-filter predecessor) · 3.4.08-resnet (where depth got even deeper) · 3.5.02-batch-normalization (the trick VGG lacked in 2014) · 3.4.15-transfer-learning · 3.6.03-receptive-field