Intuition The one core ideaBoth DenseNet and EfficientNet are recipes for building an image-recognition network out of the same handful of Lego bricks — the feature map , the convolution , and the channel . DenseNet's trick is let every layer see every earlier layer's output ; EfficientNet's trick is grow a network in three directions at once, in balance . Everything on the parent page is just those two ideas expressed in symbols — and this page earns every symbol.
Before we can read a single line of the parent note, we need a shared picture of what a CNN actually moves around. Let us build the vocabulary brick by brick, each brick resting on the one before.
Definition Pixel grid, heightH W pixels . Count the rows: that number is the height H width W 224 × 224 H = 224 W = 224
Look at the left panel of the figure below: a grey grid, H W
But a colour image is not one sheet — it is three sheets stacked: how red each pixel is, how green, how blue. Each sheet is called a channel .
C channel is one flat H × W C = 3 feature map . So a feature map is one channel of learned evidence , and a stack of them has shape H × W × C
Why does the topic need this? Every quantity the parent talks about — "64 256 1024 how many of these grids are stacked at that point in the network . When DenseNet says a layer "produces k k
Intuition Why a convolution and not a plain multiply?A face can appear top-left or bottom-right of a photo. We want a detector that finds "an eye" wherever it is . A plain fully-connected multiply learns a separate weight for every pixel position — huge and position-locked. A convolution instead learns ONE small stamp and slides it over every position, so the same pattern is found everywhere. That sharing is the whole reason CNNs exist.
Definition Kernel / filter, size3 × 3 kernel (also called a filter ) is a tiny grid of weights, usually 3 × 3 1 × 1 5 × 5
Look at the middle figure: the small magenta 3 × 3
1 × 1 1 × 1 not look at neighbours — it only mixes the C at a single spot into a new set of channels. Its whole job is to change the number of channels (e.g. squeeze 256 128 3 × 3 3.4.8-Inception-and-GoogLeNet .
Definition Depthwise convolutionA normal 3 × 3 all input channels into each output. A depthwise convolution gives each channel its own private 3 × 3 3.4.9-MobileNet-and-Depthwise-Separable-Convolutions .
Why the topic needs it: the parent's parameter counts (like 36 k 2 3 × 3 t
A parameter is one learnable number the network stores. A 3 × 3 m n m × n × 3 × 3 3 × 3 m × k × 3 × 3
Definition FLOP (floating-point operation)A FLOP is one multiply-or-add the machine performs when running the network (not stored — executed ). For one conv layer the count scales as (number of layers) × 2 × H × W FLOPs ∝ d ⋅ w 2 ⋅ r 2
Common mistake Parameters = stores . FLOPs = work it does per image. A 1 × 1
H ℓ H ℓ ℓ ℓ ell ) is just a counter: layer number 1 , 2 , 3 , … H ℓ BN → ReLU → Conv . Three sub-tools live inside it:
Definition BN = Batch NormalizationBatch Normalization rescales each channel so its numbers have a tidy spread (mean near 0 1 3.4.3-Batch-Normalization-and-Regularization .
ReLU ("rectified linear unit") is the rule ReLU ( x ) = max ( 0 , x ) 0 4 5 ∘
Definition Pooling (AvgPool / MaxPool, stride 2)Pooling shrinks a feature map by summarising each little 2 × 2 Stride 2 means "step two pixels at a time", which halves H W 112 → 56 → 28 … space ; the 1 × 1 channels — together they are the transition layer.
[ ⋅ , ⋅ ] [ x 0 , x 1 , … ] stack these feature maps on top of each other along the channel axis. If x 0 16 x 1 8 [ x 0 , x 1 ] 24
Contrast with addition (+ ResNet : adding two 16 16 merged and lost . The figure shows both: concatenation grows the stack, addition overlays it.
k k 0 k 0 k adds to the shared stack (the growth rate , e.g. 12 32 L k 0 + L k why : every layer piles on exactly k
Definition The three scaling knobsd , w , r
==depth d
==width w
==resolution r
A picture: depth = number of pancakes; width = diameter of each pancake; resolution = fineness of the batter grain.
ϕ , α , β , γ ϕ phi ) is a single dial you turn to make a bigger model (B0, B1, … B7). α , β , γ rates found by search: raise them to the power ϕ α ϕ , β ϕ , γ ϕ ≥ 1 3.4.11-Neural-Architecture-Search .
Recall Why the exponent, not a plain multiply?Turning the dial ϕ → ϕ + 1 double the FLOPs each time. Repeated doubling is exponential, so growth must be α ϕ ( α β 2 γ 2 ) ϕ α β 2 γ 2 = 2 2 ×
Convolution sliding stamp
1x1 conv changes channels
Concatenation vs addition
DenseNet and EfficientNet
State each aloud; if you cannot, re-read that section.
What does a feature map represent, and what shape does a stack of them have? One channel of learned evidence, an H × W H × W × C
Why do we use convolution instead of a full multiply? One small kernel slides over all positions, so a pattern is detected wherever it appears and weights are shared.
What is the ONLY job of a 1 × 1 To change the number of channels by mixing them at each single pixel (no neighbours).
How many parameters in a 3 × 3 m n 9 mn m ⋅ n ⋅ 3 ⋅ 3
Difference between a parameter and a FLOP? A parameter is a stored learnable number; a FLOP is one multiply/add executed while running.
What does ReLU ( x ) − 3 5 Sends − 3 → 0 5 → 5 0 x
What does stride-2 pooling do to H W Halves both by summarising each 2 × 2
Concatenation vs addition of two 16-channel stacks — resulting channel count? Concatenation → 32 → 16
After L k k 0 k 0 + L k
What do d w r α , β , γ Depth, width, resolution; constraint α ⋅ β 2 ⋅ γ 2 ≈ 2
