Foundations — VGG networks

Before you can read the parent note on VGG networks, you need to be fluent in a handful of symbols and pictures. This page builds each one from absolutely nothing. We never use a symbol before we have drawn it.

1. What is an image, to a computer?

A photo on a screen is a grid of numbers. Each little square (a pixel) holds a brightness value. A colour image holds three such grids stacked behind each other — one for Red, one for Green, one for Blue.

Why the topic needs this: every single number the parent note computes — "224×224×64", "25,088 units", memory in MB — is just counting the boxes in one of these stacked grids. If you can picture the box-stack, you can picture every activation shape in VGG.

The tall stacked grid is called a feature map once we are past the input. Same picture, but the numbers no longer mean "brightness" — they mean "how strongly some detector fired here".

2. The sliding window: a convolution

Take a small square of weights — say $3\times 3$ numbers. Slide it across the image. At each stop, multiply overlapping numbers, add them all up, write the single result into a new grid. That single sliding-add is the convolution operation.

Why the topic needs this: VGG's entire claim ("stack small filters") is a claim about kernels. You cannot understand "two 3×3 = one 5×5" without first seeing what one 3×3 slide does.

3. Stride, padding, and the output-size formula

Two dials control how the window slides.

Now we can read the parent's output-size formula. Let $H_{\text{in}}$ be the input height and $H_{\text{out}}$ the output height:

Why this exact formula? $H_{\text{in}}+2p$ is the padded height. Subtracting $k$ counts how far the window's top edge can travel before its bottom edge runs off. Dividing by $s$ counts how many jumps that is. The "$+1$" counts the starting position itself. Read it as: (number of jumps) + (the first stop).

For VGG's $3\times 3$ same-padding, $p=\lfloor 3/2\rfloor =1$ and $s=1$: $H_{\text{out}}=\lfloor \frac{H_{\text{in}}+2-3}{1}\rfloor +1=H_{\text{in}}$ Size is unchanged — that is what "same padding" means.

Edge / degenerate cases (so no scenario surprises you):

• If $p=0$ (no padding), a $3\times 3$ conv shrinks the grid by 2 each side of the count: $H_{\text{out}}=H_{\text{in}}-2$.
• If $s=2$ (pooling-style stride), the grid roughly halves: $H_{\text{out}}\approx H_{\text{in}}/2$.
• The floor only bites when $H_{\text{in}}+2p-k$ isn't divisible by $s$ — then some right-edge pixels are simply never a window centre.

4. Channels in, channels out — where $C_{in}$ and $C_{out}$ come from

A kernel doesn't just slide over one grid — it slides over the whole stack of $C_{in}$ input grids at once. So a single filter is actually a little box of weights: $k_h\times k_w\times C_{in}$.

Each such box produces one output grid. Stack $C_{out}$ different boxes → you get $C_{out}$ output grids.

Why the topic needs this: VGG's "double the channels after pooling" is literally "double $C_{out}$". And every parameter count multiplies by these two numbers.

5. Counting the weights — the parameter formula

Now every symbol in the parent's parameter formula is defined, so we can earn it.

One filter box holds $k_h\times k_w\times C_{in}$ weights, plus one bias (a single "always-added" number per filter). There are $C_{out}$ filters:

6. Receptive field — the notation ${\text{RF}}_n$ and $\prod$

The parent note's headline result ("two 3×3 see a 5×5") is about the receptive field.

The stacking formula uses a product symbol. You must not meet $\prod$ unprepared:

Why this tool and not just "add kernel sizes"? Because a later layer's one-pixel step corresponds to several input pixels once earlier strides have spread things out — the product $\prod s_i$ is exactly that spreading factor. For VGG all early strides are 1, so it stays simple. Worked, for three $3\times 3$ layers (all $s=1$): ${\text{RF}}_1=3,{\text{RF}}_2=3+2\times 1=5,{\text{RF}}_3=5+2\times 1=7$

See 3.6.03-receptive-field for the full geometric picture.

7. Pooling, ReLU, and floor — the last small pieces

Prerequisite map

Related building blocks in the vault: 3.4.01-convolution-operation (the slide itself), 3.4.05-alexnet (the large-filter network VGG improves on), 3.4.08-resnet and 3.5.02-batch-normalization (what came after), 3.6.03-receptive-field (deeper on RF), 3.4.15-transfer-learning (where VGG shines).

Equipment checklist