Visual walkthrough — LeNet and AlexNet

Both LeNet and AlexNet describe their layers with lines like "Input $32\times 32$, 6 filters of $5\times 5$ → Output $28\times 28\times 6$". Where does that 28 come from? The whole architecture — every layer size, every parameter count — rests on one small counting result. Let us build it from absolute zero: no formula assumed, just squares sliding over a grid.

We will need three plain words before any symbol:

Symbols earned: $W$ (input side), $K$ (kernel side), $S$ (stride). We will add $P$ (padding) only when we need it, in Step 5.

---

Step 1 — Lay the kernel at the top-left corner

WHAT. Put the $K\times K$ window so its top-left cell sits on the image's top-left cell. Multiply the overlapping numbers, add them up: that produces the first output value.

WHY. Every counting problem needs a starting point. The left-most, top-most legal position is the natural "position 1". If we can count how far the window travels from here, we are done.

PICTURE. The orange window covers columns $1$ through $K$ of the blue image. Look at how much room is left to its right — that leftover room is what limits how far it can travel.

Here $W=7$, $K=3$ (a toy image so we can count by eye). The window occupies columns 1–3; columns 4–7 are still free to its right.

---

Step 2 — Slide right one pixel at a time and count landings (stride 1)

WHAT. Move the window right by $S=1$ pixel repeatedly. Each landing is a new output value. We stop the instant the window's right edge would fall off the image.

WHY. We are literally counting positions. The output width is nothing more mysterious than "how many times did the window land before running out of room?"

PICTURE. Below, each colored outline is one legal landing of the $3\times 3$ window on the $7$-wide image. Count them: positions starting at column $1,2,3,4,5$ — that is 5 landings.

Now reason about the count. The window's left edge may start at column $1$ and may go as far right as column $W-K+1$ (any further and the right edge, at start-column $+K-1$, would exceed $W$). So the number of landings is

$\underset{\text{last start column}}{\underbrace{(W-K+1)}}-\underset{\text{first start column}}{\underbrace{1}}+\underset{\text{count both ends}}{\underbrace{1}}=W-K+1.$

• $W-K+1$ is the last column the left edge can sit on.
• Subtracting the first start ($1$) and adding $1$ back is the classic "fence-post" count — the number of integers from $a$ to $b$ inclusive is $b-a+1$.

Check with the picture: $7-3+1=5$. That matches the five outlines. This is exactly the parent note's "the kernel can slide $(32-5+1)=28$ positions" for LeNet's C1.

---

Step 3 — Big jumps: what stride $S$ changes

WHAT. Instead of moving $1$ pixel per step, move $S$ pixels. Fewer landings fit, so the output shrinks faster.

WHY. AlexNet's Conv1 uses stride $4$ to aggressively downsample — collapse a huge image quickly so later layers are cheap. Stride is our knob for "how coarsely do we sample positions?"

PICTURE. The window starts at column $1$, then $1+S$, then $1+2S$, ... The gaps between landings are now $S$ wide. We count how many jumps of size $S$ fit inside the travel room $W-K$.

The total distance the left edge is allowed to travel is $W-K$ (from Step 2). Each step consumes $S$ pixels of that budget, so the number of extra steps we can take is $\lfloor (W-K)/S\rfloor$. Add the very first landing (the "$+1$"):

$\text{Output}=\lfloor \frac{W-K}{S}\rfloor +1$

• $W-K$ — total travel room for the left edge (Step 2).
• $÷S$ — how many jumps of size $S$ fit in that room.
• $\lfloor \rfloor$ — the floor: round down, because a partial jump that would push the window off the edge is not a legal landing. You cannot half-land.
• $+1$ — the starting position, which took zero jumps.

---

Step 4 — Worked landings for AlexNet's Conv1 (the "224 vs 227" puzzle)

WHAT. Apply Step 3 to AlexNet: $W=224$, $K=11$, $S=4$, no padding yet.

WHY. The parent note flags a famous confusion — the paper says the output is $55\times 55$ but the plain formula gives $54$. Let us see both numbers appear and understand which input produces which.

PICTURE. The bar shows the travel budget $W-K=213$ chopped into jumps of $S=4$. It does not divide evenly — a stub of length $1$ is left over, which the floor discards.

$\text{Output}=\lfloor \frac{224-11}{4}\rfloor +1=\lfloor \frac{213}{4}\rfloor +1=\lfloor 53.25\rfloor +1=53+1=54.$

• $224-11=213$ — the travel room.
• $213/4=53.25$ — jumps including an impossible quarter-jump.
• $\lfloor 53.25\rfloor =53$ — discard the quarter; that stub is the gray leftover in the figure.

So the honest arithmetic gives 54. The paper's $55$ comes from feeding the layer a $227$-wide input (one pixel of implicit border added), which we resolve next.

---

Step 5 — Padding $P$: gluing a border so the count comes out clean

WHAT. Wrap the image in $P$ rings of extra pixels (usually zeros) on every side. The window now has more room, so more landings fit and the output grows.

WHY. Two reasons. (1) It lets you choose the output size — e.g. keep it equal to the input ("same" convolution), as AlexNet's Conv2/3/4/5 do with padding $2,1,1,1$. (2) It rescues edge pixels, which otherwise get visited by the window far less often than centre pixels.

PICTURE. The blue image gains a green border of width $P$ on each side. Its effective width becomes $W+2P$ — the "$2$" because padding is added on both the left and the right.

Replace $W$ by $W+2P$ everywhere in Step 3's count:

AlexNet resolved. With $P$ effectively giving a $227$-wide input: $\lfloor \frac{227-11}{4}\rfloor +1=\lfloor \frac{216}{4}\rfloor +1=54+1=55.✓$ The mystery "$55$" was padding hiding inside the implementation, exactly as the parent note says.

---

Step 6 — Every case, including the degenerate ones

WHAT. Push the formula to its limits so no scenario surprises you.

WHY. Contract rule: the reader must never meet an input we did not show.

• $K=W$, $S=1$, $P=0$ (window as big as image): $\lfloor (W-W)/1\rfloor +1=0+1=1$. Output is a single number. This is exactly LeNet's C5: input $5\times 5$, kernel $5\times 5$ → $1\times 1$. The window fits in one and only one spot — hence C5 "is effectively fully connected".
• $K=1$, $S=1$, $P=0$ (single-pixel window): $\lfloor (W-1)/1\rfloor +1=W$. Output = input; a $1\times 1$ convolution mixes channels but never moves spatially.
• Pooling is the same formula. LeNet's S2: $2\times 2$ pool, $S=2$ on a $28$-wide map: $\lfloor (28-2)/2\rfloor +1=13+1=14$. Matches $14\times 14$. Pooling windows count landings identically to conv windows — see Pooling Layers and Convolutional Layers.
• Non-even division ($W-K+2P$ not a multiple of $S$): the floor silently drops the leftover stub (Step 4). No error, just a slightly smaller map.
• Illegal ($K>W+2P$): numerator negative, "output $<1$" — the window does not even fit once. The layer is impossible; frameworks throw an error.

---

The one-picture summary

The entire derivation on one line: start room, minus the window, plus padding on both sides, divided into jumps, round down, add the starting landing.

Recall Feynman retelling — say it like you'd explain to a friend

Imagine parking a square car in a long thin garage. The garage is $W$ pixels long; the car is $K$ pixels long. Once the car nose is at the very back, the room left in front of it is $W-K$. If you nudge the car forward $S$ pixels each parking spot, the number of extra spots is that leftover room divided by $S$ — and you round down, because a spot that pokes the car out the door doesn't count. Then add one for the very first spot (nose against the back wall). That total number of parking spots is how wide the output image is. Padding is just building a little porch of length $P$ on both ends of the garage, so the garage is really $W+2P$ long and more spots fit. That's why LeNet's $32$-image with a $5$-car and no porch gives $32-5+1=28$ spots, and why AlexNet's $55$ only appears once you admit there was a secret one-pixel porch making the input $227$, not $224$. Same one rule builds every layer size in both networks — convolution and pooling alike.

Recall Quick self-test

LeNet C1 output side for input 32, kernel 5, stride 1, pad 0 ::: $\lfloor (32-5)/1\rfloor +1=28$ AlexNet Conv1 with the honest 224 input ::: $\lfloor (224-11)/4\rfloor +1=54$ AlexNet Conv1 with the implicit 227 input ::: $\lfloor (227-11)/4\rfloor +1=55$ LeNet S2 pooling, input 28, kernel 2, stride 2 ::: $\lfloor (28-2)/2\rfloor +1=14$ Why "$+2P$" and not "$+P$"? ::: padding is added on both left and right sides Why the floor? ::: landings are whole things; a window poking off the edge is not a legal position

Related builds: deeper stacks of this same counting rule underlie VGGNet and ResNet; the regularization tricks around it live in Dropout and Batch Normalization; see also Image Classification where these output maps finally become class scores.