Worked examples — DenseNet and EfficientNet

This page is a calculator you can trust. The parent note gave you the formulas; here we run them through every kind of input — small blocks, big blocks, compression that rounds down, the degenerate "no bottleneck" case, exponential scaling that blows up, and the exam-twist where a number is hidden. Before touching a single example, let us list every situation these formulas can face.

---

The scenario matrix

Every problem in this topic is one of the cells below. If you can do all of them, nothing on an exam can surprise you.

Cell	Case class	What makes it tricky
A	DenseNet channel counting, plain	Running concatenation total $k_0+\ell k$
B	Transition compression with clean division	$\theta m$ is a whole number
C	Transition compression that rounds down	$\lfloor \theta m\rfloor$ — the floor bites
D	Bottleneck vs. no-bottleneck parameter count	when does $4mk$ beat $9mk$? (degenerate small-$m$ edge)
E	Full network trace (multi-block, DenseNet-121 style)	chaining A+B across 4 blocks
F	EfficientNet constraint check ($\alpha {\beta }^2{\gamma }^2\approx 2$)	verifying a candidate triple
G	EfficientNet compound scaling at a given $\varphi$	exponential growth ${\alpha }^{\varphi }$, resolution rounding
H	Limiting / degenerate scaling ($\varphi =0$, one dimension only)	what "no scaling" and "unbalanced scaling" cost
I	Word problem (memory / FLOP budget)	real-world units, choosing a config
J	Exam twist (solve backwards for a hidden $k$ or $\varphi$)	invert the formula

Recall What does concatenation do to channel count that addition does not?

Concatenation ::: stacks channels, so counts add up ($k_0+\ell k$); ResNet addition keeps the channel count fixed.

Below, ten worked examples cover cells A–J. Each cell is labelled.

---

Example 1 — Cell A: plain channel counting

Forecast: guess the final number before reading. (Add $k$ four times to 16 — hold that thought.)

1. Layer 1 sees $k_0=16$, outputs $k=8$ new → running total $16+8=24$. Why this step? Each $H_{\ell }$ produces exactly $k$ feature maps; concatenation appends them.
2. Layer 2 sees 24, outputs 8 → $24+8=32$. Why this step? Its input is everything so far — the concatenation $[{\mathbf{x}}_0,{\mathbf{x}}_1]$.
3. Layer 3 sees 32 → $32+8=40$.
4. Layer 4 sees 40 → $40+8=48$.

Answer: output $=48$ channels.

Verify: closed form $k_0+Lk=16+4\cdot 8=48$. ✓ The step-by-step and the formula agree.

---

Example 2 — Cell B: compression, clean division

Forecast: half of 48 — will the floor matter here?

1. Apply $\lfloor \theta m\rfloor$ with $m=48$, $\theta =0.5$ → $\lfloor 24.0\rfloor =24$. Why this step? The transition's $1\times 1$ conv projects $m$ channels down to $\lfloor \theta m\rfloor$ to keep growth in check.

Answer: 24 channels.

Verify: $48\times 0.5=24$ exactly — no rounding needed. Units: channels → channels. ✓

---

Example 3 — Cell C: compression that rounds down

Forecast: $45/2=22.5$ — you cannot have half a channel. What happens?

1. Compute $\theta m=0.5\times 45=22.5$. Why this step? Raw product first, then floor — order matters.
2. Apply the floor: $\lfloor 22.5\rfloor =22$. Why this step? Channel count must be a whole number; DenseNet uses $\lfloor \cdot \rfloor$ (round down), never round-to-nearest.

Answer: 22 channels (not 23!).

Verify: $\lfloor 22.5\rfloor =22$. ✓

---

Example 4 — Cell D: bottleneck vs. no-bottleneck (degenerate small-$m$)

Forecast: bottlenecks are famous for saving parameters — do they always?

Recall the two costs from the parent:

• With bottleneck: $4mk$ (the $1\times 1$) $+36k^2$ (the $3\times 3$ on $4k$ channels).
• Without bottleneck: $9mk$ (a plain $3\times 3$).

1. Late block, $m=300$, $k=12$:
   • With: $4\cdot 300\cdot 12+36\cdot 12^2=14400+5184=19584$.
   • Without: $9\cdot 300\cdot 12=32400$. Why this step? Plug $m$ into both formulas and compare totals. → Bottleneck saves $32400-19584=12816$ parameters. Worth it.
2. Early degenerate, $m=12$, $k=12$:
   • With: $4\cdot 12\cdot 12+36\cdot 12^2=576+5184=5760$.
   • Without: $9\cdot 12\cdot 12=1296$. Why this step? Same formulas, tiny $m$. Now the fixed $36k^2$ term dominates. → Bottleneck is worse by $5760-1296=4464$ params.

Answer: bottlenecks help when $m$ is large (deep in a block); at tiny $m$ the constant $36k^2$ overhead makes them lose.

Verify: solve the break-even $4mk+36k^2=9mk\Rightarrow 36k^2=5mk\Rightarrow m=\frac{36k}{5}$. For $k=12$, $m=86.4$. So above $m\approx 87$ channels the bottleneck wins — consistent with "worse at 12, better at 300." ✓

---

Example 5 — Cell E: full DenseNet-121-style trace

Forecast: the parent quotes 256 → 128 → 512 → 256 → 1024 → 512 → 1024. Reproduce it.

1. Block 1 out: $64+6\cdot 32=256$. Why? Cell-A formula with $k_0=64,L=6$.
2. Transition 1: $\lfloor 0.5\cdot 256\rfloor =128$. Why? Cell-B compression.
3. Block 2 out: $128+12\cdot 32=512$.
4. Transition 2: $\lfloor 0.5\cdot 512\rfloor =256$.
5. Block 3 out: $256+24\cdot 32=1024$.
6. Transition 3: $\lfloor 0.5\cdot 1024\rfloor =512$.
7. Block 4 out: $512+16\cdot 32=1024$ (no transition follows → global average pool). Why this step for each? Alternate Cell A (block) and Cell B (transition) down the chain.

Answer: $256\to 128\to 512\to 256\to 1024\to 512\to 1024$.

Verify: every arrow matches the parent's DenseNet-121 table. ✓

---

Example 6 — Cell F: EfficientNet constraint check

Forecast: the whole point is FLOPs doubling per $\varphi$-step — is this triple close to 2?

1. Compute ${\beta }^2=1.1^2=1.21$. Why? Width appears squared in FLOPs (input × output channels).
2. Compute ${\gamma }^2=1.15^2=1.3225$. Why? Resolution squared (spatial $H\times W$).
3. Multiply: $1.2\times 1.21\times 1.3225=1.9203$. Why this step? This is $(\alpha {\beta }^2{\gamma }^2)$ — the per-$\varphi$ FLOP multiplier.

Answer: $\approx 1.92$, which rounds to $2$ within the "$\approx$" tolerance. ✓ Valid.

Verify: $1.2\cdot 1.21\cdot 1.3225=1.9203$, and $|1.9203-2|=0.0797<0.1$. ✓

---

Example 7 — Cell G: compound scaling at $\varphi =6$ (B7)

Forecast: exponentials at $\varphi =6$ — expect roughly $3\times$ depth.

1. Depth $={\alpha }^{\varphi }=1.2^6$. Why? Depth scales as ${\alpha }^{\varphi }$ by definition of compound scaling. $1.2^6=2.9860\approx 3.0\times$.
2. Width $={\beta }^{\varphi }=1.1^6=1.7716\approx 1.77\times$. Why? Width scales as ${\beta }^{\varphi }$.
3. Resolution $={\gamma }^{\varphi }\times 224=1.15^6\times 224$. $1.15^6=2.3131$, so $2.3131\times 224=518.1$. Why this step? Resolution scales as ${\gamma }^{\varphi }$ off the 224 baseline.

Answer: depth $\approx 3.0\times$, width $\approx 1.77\times$, resolution $\approx 518$ px.

Verify: $1.2^6=2.9860$, $1.1^6=1.7716$, $1.15^6\cdot 224=518.1$. ✓

---

Example 8 — Cell H: limiting / degenerate scaling

Forecast: $\varphi =0$ should mean "do nothing." And pouring all budget into depth should waste it.

1. $\varphi =0$: $d={\alpha }^0=1$, $w={\beta }^0=1$, $r={\gamma }^0=1$. Why? Anything to the power 0 is 1 — this is the B0 baseline, no scaling.
2. Depth-only $2\times$: FLOPs $\propto d\cdot w^2\cdot r^2=2\cdot 1\cdot 1=2$. Cost = 2× FLOPs. Why this step? Depth is linear in FLOPs.
3. Balanced $2\times$ FLOPs (i.e. $\varphi =1$): by construction $(\alpha {\beta }^2{\gamma }^2{)}^1=2$ FLOPs too — same cost, but spread across three dimensions. Why this step? The constraint was designed so one $\varphi$-step = 2× FLOPs.

Answer: (a) $\varphi =0$ → the untouched baseline. (b) Both spend 2× FLOPs, but balanced scaling empirically gives higher accuracy — same budget, better spent. The degenerate depth-only choice is legal but suboptimal.

Verify: ${\alpha }^0={\beta }^0={\gamma }^0=1$; and $(\alpha {\beta }^2{\gamma }^2{)}^1=1.9203\approx 2$, matching the depth-only 2×. ✓

---

Example 9 — Cell I: word / budget problem (with figure)

Forecast: solve $24+16L\le 100$ for the biggest whole $L$.

1. Set inequality: $24+16L\le 100$. Why this step? Output channels grow as $k_0+Lk$; the cap is a hard memory limit.
2. Solve: $16L\le 76\Rightarrow L\le 4.75$. Why this step? Isolate $L$.
3. Floor to whole layers: $L=4$ (since $L=5$ gives $24+80=104>100$). Why this step? You cannot have a fractional layer, and 5 overshoots.
4. Actual output: $24+4\cdot 16=88$ channels. Compress: $\lfloor 0.5\cdot 88\rfloor =44$.

Answer: $L=4$, block output $88$, post-transition $44$ channels.

The figure shows the running channel total climbing by $k=16$ each layer (burnt-orange steps), the plum ceiling at 100, and the teal drop to 44 after compression.

Verify: $L=5$ gives $104>100$ (rejected); $L=4$ gives $88\le 100$ (accepted); $\lfloor 44.0\rfloor =44$. ✓

---

Example 10 — Cell J: exam twist (solve backwards)

Forecast: invert each formula instead of plugging forward.

1. Channel inversion: $k_0+Lk=200\Rightarrow 40+10k=200\Rightarrow 10k=160\Rightarrow k=16$. Why this step? The output formula is linear in $k$; algebra recovers it.
2. Scaling inversion: ${\alpha }^{\varphi }=1.728$ with $\alpha =1.2$. Take logs: $\varphi =\frac{\ln 1.728}{\ln 1.2}$. Why this step? $\varphi$ is the exponent; the logarithm is the tool that "undoes" an exponent (it answers "what power gives this value?", exactly like arctan undoes tan).
3. Evaluate: $\ln 1.728=0.5470$, $\ln 1.2=0.1823$, ratio $=3.0$. Why this step? $1.2^3=1.728$, so $\varphi =3$ cleanly.

Answer: $k=16$, and $\varphi =3$.

Verify: forward-check both: $40+10\cdot 16=200$ ✓; $1.2^3=1.728$ ✓.

---

Recall Why is the logarithm the right tool to recover

$\varphi$ from ${\alpha }^{\varphi }$? Because ::: $\varphi$ sits in the exponent, and $\log$ is the exact inverse of exponentiation — it asks "which power produced this number?"

Connections

• Prerequisite for the concatenation-vs-addition contrast: 3.4.7-ResNet-and-Skip-Connections
• MBConv depthwise blocks used by EfficientNet: 3.4.9-MobileNet-and-Depthwise-Separable-Convolutions
• Where $\alpha ,\beta ,\gamma$ come from: 3.4.11-Neural-Architecture-Search
• Bottleneck & $1\times 1$ heritage: 3.4.8-Inception-and-GoogLeNet
• The BN in every $H_{\ell }$: 3.4.3-Batch-Normalization-and-Regularization
• Shrinking these models further: 4.2.5-Model-Compression-and-Pruning