VGG networks: Question bank — AI-ML deep dive

Definition Symbol glossary — read this first

Every symbol used on this page, in plain words:

$k$ — the kernel (filter) size along one side, e.g. a " $3 \times 3$ conv" has $k = 3$ . VGG's kernels are square, so height and width are both $k$ ; we write $k_{h}$ and $k_{w}$ only when a formula treats them separately, but for VGG $k_{h} = k_{w} = k$ .
$s$ — the stride, how many pixels the filter jumps between positions. $s = 1$ = slide by one pixel; $s = 2$ = skip every other position.
$p$ — the padding, how many rows/columns of zeros we glue around each edge of the input before convolving.
$C$ — the number of channels (feature maps). $C_{in}$ = channels going into a layer, $C_{o u t}$ = channels coming out. When both equal the same value we just write $C$ .
$H_{in}$ , $H_{o u t}$ — the height (in pixels) of the feature map before and after a layer. Width behaves identically, so we only track height.
$RF$ (receptive field) — the size, in original-input pixels, of the region one output pixel can "see". A $3 \times 3$ conv has $RF = 3$ : each output pixel looks at a $3 \times 3$ input window.
$P$ — the number of learnable parameters (weights + biases) in a layer.
AlexNet — the 2012 CNN that first won ImageNet at scale, using large early filters ( $11 \times 11$ , $5 \times 5$ ); VGG's whole pitch is "replace those with stacks of $3 \times 3$ ." See 3.4.05-alexnet.
VGG "configuration C" — one experimental variant in the original VGG paper that inserted a few $1 \times 1$ convolutions; it is not the famous VGG-16/19, which use $3 \times 3$ only. We mention it purely so you can recognise the trap.

Definition Where the parameter formula comes from (so the "+1" isn't magic)

A convolution slides one filter over the image; that filter is a little block of weights of shape $k_{h} \times k_{w} \times C_{in}$ — it must have a weight for every pixel in its window ( $k_{h} \times k_{w}$ ) and for every input channel ( $C_{in}$ ) it reads. Multiply those out: $k_{h} \times k_{w} \times C_{in}$ weights per filter.

Now, after summing all those weighted inputs, a neuron adds one constant offset — its bias — which lets it shift its activation threshold without changing any input weight. One filter produces one output channel, and it carries exactly one such bias. That is the " $+ 1$ ": $weights per filter k_{h} \times k_{w} \times C_{in} + its single bias 1$ Finally, a layer with $C_{o u t}$ output channels is just $C_{o u t}$ independent filters, each with its own weight block and its own bias — so multiply the whole bracket by $C_{o u t}$ : $P = (k_{h} \times k_{w} \times C_{in} + 1) \times C_{o u t}$ The bias sits inside the bracket precisely because there is one per filter, not one for the whole layer.

Formula Receptive-field recurrence — derived, not just stated

Claim: stacking convs grows the receptive field by $RF_{n} = RF_{n - 1} + (k_{n} - 1) \times \prod_{i = 1}^{n - 1} s_{i}$ Why, step by step (follow the figure below alongside):

A single layer- $n$ output pixel reads $k_{n}$ neighbouring pixels of layer $n - 1$ . Going from the centre tap to an edge tap moves $(k_{n} - 1)$ positions on the layer- $(n - 1)$ grid — that is why the added term starts with $(k_{n} - 1)$ , not $k_{n}$ .
But one "position" on the layer- $(n - 1)$ grid is not one original-input pixel. Every earlier layer with stride $s_{i}$ spread out its grid: after a stride-2 layer, adjacent cells are 2 original pixels apart, after two stride-2 layers, 4 apart, and so on. So one step now covers $\prod_{i = 1}^{n - 1} s_{i}$ original pixels — the stride product.
Multiply the $(k_{n} - 1)$ extra steps by how many original pixels each step is worth $(\prod s_{i})$ , and add that reach onto the region layer $n - 1$ already saw ( $RF_{n - 1}$ ).
Base case: a single $3 \times 3$ conv sees $RF_{1} = 3$ — three input pixels wide.

Because VGG keeps stride 1 inside every block, the stride product stays 1, so each extra $3 \times 3$ simply adds $(3 - 1) \times 1 = 2$ : $3 \to 5 \to 7$ for one, two, three stacked convs.

True or false — justify

Every item: decide true or false, then give the one-sentence reason. The reason is the answer — a right verdict with a wrong reason is a miss.

Two stacked 3×3 convolutions and one 5×5 convolution see exactly the same input patch per output pixel.

True — the receptive field (

RF

, the input region one output pixel sees) of both is

5 \times 5

(using

RF_{2} = 3 + (3 - 1) \times 1 = 5

), but the two-3×3 stack is not the same function because it has a ReLU in the middle, so it can represent things a single linear 5×5 kernel cannot.

Because two 3×3 convs equal one 5×5 in receptive field, they also have the same number of parameters.

False — with

C

input =

C

output channels, two 3×3 layers cost

18 C^{2}

weights versus

25 C^{2}

for the 5×5 (28% fewer); equal reach does not mean equal parameter count.

VGG keeps the spatial height and width constant inside a block.

True — VGG uses same-padding (padding

p = 1

for a 3×3 conv), so

H_{out} = H_{in}

; only the 2×2 max-pool between blocks halves the size.

Doubling the channels after every pool keeps the total number of activations constant.

False — 2×2 pooling cuts activations by 4× and doubling channels only multiplies by 2, so the count still drops to half (

\frac{H W C}{2}

, where

H, W

are height/width and

C

is channel count); doubling softens the loss, it does not cancel it.

Most of VGG-16's 138M parameters live in the convolutional layers.

False — the three fully-connected layers (especially the

25, 088 \times 4096

first FC) hold the large majority; the conv stack is comparatively lean.

VGG-19 always beats VGG-16 on accuracy because it is deeper.

False — extra depth gives diminishing returns and the gain is tiny-to-none on ImageNet; "deeper" only helps until optimization difficulty (pre-ResNet, no skip connections) eats the benefit.

Replacing the 5×5 filters of AlexNet with stacked 3×3 filters increases the number of non-linearities.

True — one 5×5 = one ReLU, whereas two 3×3 = two ReLUs, giving more places to bend the decision surface for the same effective receptive field.

VGG-16 was initialized by copying weights from a trained 11-layer VGG.

False — the deeper VGG-16/19 configurations were in fact trained from scratch; the authors report using the shallow VGG-11 to initialize the first and last layers of some configs, but VGG-16/19 converged fine from random init using careful weight scaling — so "always bootstrapped from VGG-11" is a myth. The real 2014 difficulty was optimizing deep nets before batch normalization, solved by good initialization and small (3×3) filters.

Spot the error

Each statement contains one wrong claim. Name it and correct it.

"VGG-16 uses 3×3 convolutions everywhere and 1×1 convolutions to match channel dimensions across blocks."

Wrong — canonical VGG-16/19 use only 3×3 convs; channel changes come from the chosen output-channel count

C_{o u t}

of the 3×3 layer itself. (The 1×1 idea belongs to the experimental "configuration C" defined in the glossary, not the famous nets.)

"After the last pooling the feature map is 7×7×512, so flattening gives 7×512 = 3584 units into the first FC layer."

Wrong arithmetic — flatten multiplies all three dimensions:

7 \times 7 \times 512 = 25, 088

units, not

7 \times 512

"Same-padding for a 3×3 stride-1 conv means padding of 3 on each side."

Wrong — same-padding is

p = ⌊ k /2 ⌋ = 1

for kernel size

k = 3

; padding of 3 would enlarge the map, not preserve it.

"Three stacked 3×3 convs give a 9×9 receptive field because 3+3+3 = 9."

Wrong — you add

(k - 1) = 2

per extra layer, not the full kernel:

3 + 2 + 2 = 7

, so it matches a single 7×7.

"Doubling channels after pooling increases spatial resolution to compensate for the pool."

Wrong — doubling adds channels

C

(feature richness), not spatial resolution

H \times W

; the pool has already halved height and width and nothing restores that.

"The +1 in

P = (k_{h} k_{w} C_{in} + 1) C_{o u t}

is one shared bias for the whole layer."

Wrong — it is one bias per output channel (that is why the whole bracket, including the

+ 1

, is multiplied by

C_{o u t}

), so a 64-filter layer has 64 biases, not 1.

"Because VGG uses the same 3×3 kernel throughout, every layer has the same number of parameters."

Wrong — parameter count

P

scales with

C_{in} \times C_{o u t}

, which grows as channels double each block; a 512→512 layer has vastly more weights than a 3→64 one despite the identical 3×3 kernel.

Why questions

Give the reason, not the restated fact.

Why prefer two 3×3 convs over one 5×5 even when both cover the same area?

Fewer parameters (

18 C^{2}

25 C^{2}

for

C

-in/

C

-out layers) and an extra ReLU for more expressive power — you spend less and buy more non-linearity.

Why does VGG start with only 64 channels (

C_{o u t} = 64

) instead of jumping straight to 512?

Low-level features (edges, colors) need few distinct detectors, so a wide first layer would waste computation at full 224×224 resolution where activations are most expensive.

Why does VGG double channels specifically after pooling rather than at random layers?

Pooling quarters the spatial cost, freeing a compute budget to widen channels; doubling right there keeps the per-layer FLOP/memory profile roughly balanced across blocks.

Why does VGG stop doubling at 512 rather than continuing to 1024, 2048?

Diminishing returns — memory and parameters explode while deep semantic features do not need proportionally more width, so extra channels buy little accuracy.

Why is aggressive data augmentation essential for VGG?

138M parameters on ~1.2M images invites severe overfitting; random crops, flips and color jitter effectively multiply the dataset, acting as a regularizer.

Why can VGG still be the better choice for transfer learning on a small dataset despite being "obsolete"?

Its simple, linear feature hierarchy transfers cleanly and is easy to fine-tune/interpret, so on <10k images its rich generic features often beat efficient-but-tangled architectures.

Why does uniform same-padding matter for VGG's design clarity?

It guarantees the spatial size only ever changes at pools, so you can reason about each block's resolution by counting pools alone — no per-conv bookkeeping.

Why do 1×1 kernels fail to do the job VGG's 3×3 kernels do?

A 1×1 kernel mixes only channels at a single pixel — it has no spatial neighborhood, so it cannot detect edges or local texture the way a 3×3 window can.

Edge cases

Boundary and degenerate scenarios the neat formulas (parameter count, output size, and the receptive-field recurrence — all defined at the top of the page) don't advertise.

What happens to

H_{out} = ⌊(H_{in} + 2 p - k) / s ⌋ + 1

if you use a 3×3 conv (

k = 3

s = 1

) with zero padding (

p = 0

Each conv shrinks the map by 2 (

H_{o u t} = H_{in} - 2

); stacked deep, the feature map would vanish — which is exactly why VGG insists on same-padding (

p = 1

Does adding same-padding change the receptive-field size of a stacked conv?

No — padding only supplies border pixels so edge outputs exist; the

(k - 1) \times \prod s_{i}

reach per layer is unchanged, so RF depends only on kernels and strides, never on

p

. (Padding shifts which input a pixel maps to, not how wide the seen window is.)

What is the receptive field of a single 3×3 conv, the base case of the recurrence?

RF_{1} = 3

; the recurrence

RF_{n} = RF_{n - 1} + (k - 1) \prod s_{i}

needs this starting value or it has nothing to build on.

Why does the stride product

\prod_{i = 1}^{n - 1} s_{i}

multiply the

(k - 1)

term in the recurrence?

Because each new layer's filter steps across the previous layer's grid, and every earlier stride has already stretched how far one such step reaches back in original-input pixels — a step of 1 on a grid that was downsampled by

s

covers

s

original pixels, so the reaches compound multiplicatively (see the orange spacing arrow in the figure).

If every stride in the recurrence were 2 instead of 1, would three 3×3 convs still give a 7×7 receptive field?

No — with stride 2 the stride product grows, so

RF = 3 + (3 - 1) \cdot 2 + (3 - 1) \cdot 4 = 3 + 4 + 8 = 15

; VGG's small clean 7×7 field relies on stride-1 convs.

What is the receptive field over the original input after two stride-2 pooling stages (each with a 3×3 conv before it)? Give the number.

Work the recurrence with strides

(1, 2, 1, 2)

for the four layers, base

RF_{1} = 3

: layer 2 (running product

s_{1} = 1

)

= 3 + 2 \cdot 1 = 5

; layer 3 (running product

s_{1} s_{2} = 1 \cdot 2 = 2

)

= 5 + 2 \cdot 2 = 9

; layer 4 (running product

s_{1} s_{2} s_{3} = 1 \cdot 2 \cdot 1 = 2

)

= 9 + 2 \cdot 2 = 13

— so a pixel deep in block 3 sees a 13×13 original-image window.

C_{in} = C_{o u t} = C

, at what

C

do the two-3×3 and one-5×5 parameter counts cross?

They never cross for

C > 0

—

18 C^{2} < 25 C^{2}

for all positive

C

, so the two-3×3 stack is always cheaper in weights, regardless of channel width.

What breaks if you feed VGG-16 a 200×200 image instead of 224×224?

The conv/pool stack still runs (it is size-agnostic), but the flattened size entering the first FC layer changes, so the fixed

25, 088 \times 4096

weight matrix no longer fits — the classifier assumes a 7×7×512 map.

Recall One-line recap of the traps

Equal receptive field ≠ equal parameters ≠ equal function; the "+1" is per-channel bias; flatten multiplies all dims; channel-doubling softens (never cancels) the pool's activation loss; canonical VGG is 3×3-only; same-padding keeps size constant but does not change RF.