VGG networks: Worked examples — AI-ML deep dive

This page takes the formulas from the VGG parent note and forces them through every kind of input they can meet: normal cases, the smallest-possible ("degenerate") cases, the zero-padding edge, huge limiting cases, a word problem, and an exam twist designed to catch you.

Before we compute anything, let's name the three machines we will keep feeding numbers into. Each is stated in plain words first.

Definition The three formulas, in words

1. Output size — "If an image goes into a conv/pool layer, how wide does it come out?" $H_{out} = ⌊ \frac{H _{in} + 2 p - k}{s} ⌋ + 1$ Here $H_{in}$ is input height in pixels, $k$ is the kernel (window) width, $p$ is padding (extra pixels glued around the border), $s$ is stride (how far the window jumps each step). The $⌊ \cdot ⌋$ ("floor") means round down to the nearest whole number, because you cannot place half a window.

2. Parameter count — "How many learnable numbers does one conv layer store?" $P = (k_{h} \times k_{w} \times C_{in} + 1) \times C_{out}$ $C_{in}$ = input channels (depth of the stack coming in), $C_{out}$ = output channels (how many filters), and the $+ 1$ is one bias per filter. See 3.4.01-convolution-operation if "channels" is fuzzy.

3. Receptive field (RF) — "How big a patch of the original image does one deep neuron ultimately look at?" $RF_{n} = RF_{n - 1} + (k_{n} - 1) \times \prod_{i = 1}^{n - 1} s_{i}$ Read $\prod$ as "multiply together"; it multiplies all the earlier strides. More at 3.6.03-receptive-field.

The scenario matrix

Every worked example below is tagged with the cell it covers. The goal: no reader ever meets a case we skipped.

Cell	Case class	What makes it tricky	Example
A	Standard conv output + params	the everyday VGG layer	Ex 1
B	Degenerate kernel $k = 1$	smallest kernel, no spatial mixing	Ex 2
C	Zero padding ( $p = 0$ , "valid")	border pixels get eaten, size shrinks	Ex 3
D	Pooling (not conv)	kernel = stride = 2, no parameters	Ex 4
E	Deep stack RF, limiting/growth	how RF grows across many layers	Ex 5
F	Real-world word problem	memory/RAM budget for a batch	Ex 6
G	Exam twist: stacked-vs-big trade	the classic "two 3×3 = one 5×5" trap	Ex 7
H	Odd input / floor rounding bites	non-divisible size, floor matters	Ex 8

Worked examples

Worked example Ex 1 — Cell A: the everyday VGG conv layer

Statement. Block-3 of VGG-16: input $56 \times 56 \times 128$ , a Conv3-256 with same-padding, stride 1. Find (a) output spatial size, (b) parameter count.

Forecast: guess before reading — will the width stay 56 or shrink? Will params be closer to 100k or 300k?

Step 1. Output size uses formula 1 with $k = 3$ , $s = 1$ , same-padding $p = ⌊ 3/2 ⌋ = 1$ . Why this step? Same-padding is defined precisely so the border is refilled and size is preserved — we verify that, not assume it. $H_{out} = ⌊ \frac{56 + 2 ( 1 ) - 3}{1} ⌋ + 1 = ⌊ 55 ⌋ + 1 = 56$ Step 2. Parameters use formula 2 with $k_{h} = k_{w} = 3$ , $C_{in} = 128$ , $C_{out} = 256$ . Why this step? Each of the 256 filters is a full $3 \times 3 \times 128$ block of weights plus 1 bias — that is what the network stores. $P = (3 \times 3 \times 128 + 1) \times 256 = (1152 + 1) \times 256 = 1153 \times 256 = 295, 168$

Verify: Output depth = $C_{out} = 256$ , so full output tensor is $56 \times 56 \times 256$ ✓. Sanity on params: dominant term $9 \times 128 \times 256 = 294, 912$ , the $+ 256$ biases nudge it to $295, 168$ — matches the parent note's number.

Worked example Ex 2 — Cell B: the degenerate

$1 \times 1$ kernel Statement. A Conv1-256 (a $1 \times 1$ conv) with input $56 \times 56 \times 128$ , stride 1, $p = 0$ . Output size and params? Also: does the spatial resolution change?

Forecast: $1 \times 1$ looks silly — does it even see neighbours?

Step 1. Output size, formula 1 with $k = 1$ , $p = 0$ , $s = 1$ . Why this step? With $k = 1$ there is nothing to overhang the border, so we expect zero shrink even with no padding — worth checking the floor formula agrees. $H_{out} = ⌊ \frac{56 + 0 - 1}{1} ⌋ + 1 = ⌊ 55 ⌋ + 1 = 56$ Step 2. Params, formula 2 with $k_{h} = k_{w} = 1$ . $P = (1 \times 1 \times 128 + 1) \times 256 = (128 + 1) \times 256 = 129 \times 256 = 33, 024$ Step 3. Interpretation. A $1 \times 1$ conv mixes the 128 channels at a single pixel — it re-weights features but never looks sideways. That is exactly why the parent note stresses canonical VGG avoids $1 \times 1$ : no spatial mixing means no edge/texture learning.

Verify: Spatial size unchanged (56→56) ✓; params much smaller than the $3 \times 3$ case (33k vs 295k), as expected since we dropped the $9 \times$ spatial factor to $1 \times$ ✓.

Worked example Ex 3 — Cell C: zero padding ("valid") shrinks the image

Statement. Same $56 \times 56 \times 128$ input, a $3 \times 3$ conv but with $p = 0$ (valid padding), stride 1. What size comes out, and how many border pixels were lost?

Forecast: With no padding the window can't sit on the corner — guess how many rows/cols vanish.

Look at the figure: with

p = 0

the

3 \times 3

window's centre can never reach the outermost ring, so that ring is dropped.

Step 1. Formula 1, $k = 3$ , $p = 0$ , $s = 1$ . Why this step? This isolates the effect of padding alone — everything else matches Ex 1. $H_{out} = ⌊ \frac{56 + 0 - 3}{1} ⌋ + 1 = ⌊ 53 ⌋ + 1 = 54$ Step 2. Loss = $56 - 54 = 2$ pixels of height (one on each side); same for width. Why this step? Confirms the " $k - 1$ " rule: a $k \times k$ valid conv eats $k - 1$ pixels total per axis.

Verify: $k - 1 = 3 - 1 = 2$ ✓. This is precisely why VGG chooses same-padding: stacking 13 valid convs would erode a 224 image down badly, wrecking the clean "halve only at pooling" design.

Worked example Ex 5 — Cell E: how the receptive field grows in a deep stack

Statement. Stack three $3 \times 3$ convs (all stride 1, before any pooling). Compute $RF_{1}, RF_{2}, RF_{3}$ . Then, as a limiting check, what would $n$ such layers give?

Forecast: parent note says three $3 \times 3$ = one $7 \times 7$ . Guess the pattern of growth.

The figure stacks the windows: each new

3 \times 3

layer widens the seen patch by exactly 2 (one pixel each side).

Step 1. Layer 1: a single conv sees its own kernel. $RF_{1} = 3$ . Why this step? The recursion needs a base case — the first layer's RF is just $k$ . Step 2. Layer 2, formula 3 with $k_{2} = 3$ and all earlier strides $= 1$ (so $\prod = 1$ ): $RF_{2} = 3 + (3 - 1) \times 1 = 5$ Step 3. Layer 3: $RF_{3} = 5 + (3 - 1) \times 1 = 7$ Why this step? Confirms three $3 \times 3$ = $7 \times 7$ coverage — the core VGG argument. Step 4 (limit). With stride-1 the product is always 1, so each layer adds $k - 1 = 2$ : $RF_{n} = 3 + 2 (n - 1) = 2 n + 1$ As $n \to \infty$ , RF grows linearly — deep stacks slowly widen the view. (Pooling, with stride 2, would make the product jump and RF grow much faster — but that is a different cell.)

Verify: Plug $n = 1, 2, 3$ into $2 n + 1$ : $3, 5, 7$ ✓. Matches the step-by-step recursion ✓.

Worked example Ex 6 — Cell F: real-world word problem (RAM budget)

Statement. You run VGG-16 and want the activation memory of the Block-3 output $56 \times 56 \times 256$ for a batch of 32 images, float32 (4 bytes each). Does it fit in a 512 MB budget?

Forecast: one image is ~3 MB — will 32 of them stay under half a gigabyte?

Step 1. Elements in one activation map. Why this step? Memory = (number of values) × (bytes per value); first we count values. $56 \times 56 \times 256 = 802, 816 values$ Step 2. Bytes for one image. $802, 816 \times 4 = 3, 211, 264 bytes \approx 3.06 MB$ Step 3. Batch of 32. Why this step? Batches multiply memory linearly — the crux of the budget question. $3, 211, 264 \times 32 = 102, 760, 448 bytes \approx 98 MB$ Step 4. Compare to budget. $98 MB < 512 MB \Rightarrow fits.$

Verify: Parent note quotes "~3.2 MB per image, ~102 MB for batch 32" — our $3.06$ MB and $98$ MB agree up to the MB-vs-MiB convention (they used $1 0^{6}$ bytes = 1 MB, giving 3.2 and 102.8). Same physics ✓. Either way it fits under 512 MB ✓.

Worked example Ex 7 — Cell G: exam twist — the stacked-vs-big-kernel trade

Statement. Prove numerically, for $C = 64$ channels, that two $3 \times 3$ convs (both $64 \to 64$ ) beat one $5 \times 5$ ( $64 \to 64$ ) on parameters — and state the two bonuses. (Weights only, ignore biases as the parent does.)

Forecast: guess the percentage saved before computing.

Step 1. One $5 \times 5$ layer, weights only. Why this step? This is the "big kernel" baseline both cover the same $5 \times 5$ receptive field (from Ex 5 logic, two $3 \times 3$ = $5 \times 5$ ). $P_{5 \times 5} = 5 \times 5 \times 64 \times 64 = 25 \times 4096 = 102, 400$ Step 2. Two $3 \times 3$ layers. $P_{2 \times 3 \times 3} = 2 \times (3 \times 3 \times 64 \times 64) = 2 \times 9 \times 4096 = 18 \times 4096 = 73, 728$ Step 3. Percent saved. Why this step? The exam wants the number, not just "fewer". $\frac{102 , 400 - 73 , 728}{102 , 400} = \frac{28 , 672}{102 , 400} = 0.28 = 28%$ Step 4. Bonuses: (i) two ReLUs instead of one → more non-linear decision surface; (ii) same $5 \times 5$ receptive field. See 3.4.05-alexnet for the big-kernel design VGG replaced.

Verify: In the general $C^{2}$ form: $25 C^{2}$ vs $18 C^{2}$ , saving $7/25 = 28%$ ✓ — independent of $C$ , so our $C = 64$ number matches the abstract claim ✓.

Worked example Ex 8 — Cell H: odd input where the floor actually bites

Statement. A pooling layer $k = 2, s = 2, p = 0$ receives an odd input $27 \times 27 \times 128$ (this can happen in non-VGG nets or with different padding). What comes out? Contrast with the even case.

Forecast: 27 is odd — does the window fall off the edge, and does the floor round up or down?

Step 1. Formula 1, $k = 2, s = 2, p = 0$ , $H_{in} = 27$ . Why this step? The floor is invisible when inputs are even (Ex 4); an odd input is where it finally changes the answer. $H_{out} = ⌊ \frac{27 + 0 - 2}{2} ⌋ + 1 = ⌊ \frac{25}{2} ⌋ + 1 = ⌊ 12.5 ⌋ + 1 = 12 + 1 = 13$ Step 2. Interpretation: the last lone column/row (pixel 27) has no partner to pool with, so it is dropped — the floor discards it. Output $13 \times 13 \times 128$ . Why this step? This is exactly why VGG keeps inputs powers-of-2-friendly ( $224 \to 112 \to 56 \to 28 \to 14 \to 7$ ): every stage stays even until 7, avoiding silent pixel loss.

Verify: Even-case check: $H_{in} = 28 \Rightarrow ⌊ 26/2 ⌋ + 1 = 13 + 1 = 14$ , clean halving ✓. Odd 27 gives 13 (not 14) — floor rounded down, confirming a pixel was dropped ✓. Compare with 3.4.08-resnet which handles size changes via strided convs instead.

Recall

Same-padding output for $3 \times 3$ stride 1 ::: $H_{out} = H_{in}$ (size unchanged) Params of Conv3-256 on 128-channel input ::: $(3 \times 3 \times 128 + 1) \times 256 = 295, 168$ Parameters in any max-pool layer ::: $0$ — it just takes the max, nothing learned Receptive field of $n$ stacked $3 \times 3$ stride-1 convs ::: $2 n + 1$ Weight saving of two $3 \times 3$ vs one $5 \times 5$ ::: $28%$ (plus an extra ReLU) Why VGG keeps sizes even down to 7 ::: so pooling's floor never silently drops a border pixel Where most of VGG-16's 138M params live ::: the fully-connected classifier, not the convs

Related: 3.5.02-batch-normalization (why deep VGG was hard to train pre-BN), 3.4.15-transfer-learning (where VGG's rich features still win), and the Hinglish version.