Docker — images, containers, Dockerfile, docker-compose: Visual walkthrough

We build up to the parent's central pipeline:

$Dockerfile docker build Image docker run Container$

but we will earn every arrow. First, some plain-word groundwork.

See Linux Namespaces and cgroups for how the isolation is actually enforced under the hood; here we care about the layer machinery.

Step 1 — Start from one instruction, get one sheet

WHAT. We take the very first line of a Dockerfile, FROM ubuntu, and ask: what does it physically produce?

WHY. Before we can stack anything, we must understand a single unit. The parent said "each instruction produces a new cached image layer." We test that claim on the smallest possible case: one instruction.

PICTURE. Look at the red sheet. FROM ubuntu downloads a ready-made filesystem — a whole minimal Ubuntu folder tree — and freezes it as layer 1. The red highlight marks the only new object created by this one line.

Figure — Docker — images, containers, Dockerfile, docker-compose

The layer is:

$layer 1 L_{1} = what changed diff (empty start \emptyset \to after FROM ubuntu files)$

$L_{1}$ — the sheet we just made.
$\emptyset$ — the empty set: before FROM, there are literally no files.
The arrow — "goes to"; it reads from empty, to a full Ubuntu tree.
So $L_{1}$ = "everything Ubuntu added", which here is the whole base.

One instruction → one read-only sheet. Claim confirmed on the base case.

Step 2 — Each new instruction adds exactly one sheet on top

WHAT. We add the next two lines and watch the stack grow.

FROM ubuntu                  # -> L1
RUN apt-get install python   # -> L2
RUN pip install flask        # -> L3

WHY. We need to see that layers accumulate in order and that each sheet only records its own change, not the whole filesystem. This is what makes them small and shareable.

PICTURE. Three stacked sheets. The red sheet ( $L_{3}$ ) is the newest change — flask's files only. It does not re-store Python or Ubuntu; those are already below it. To "see" the full filesystem you look down through all the transparent sheets at once.

The image after three steps is the ordered stack:

$Image = L_{1} ≺ L_{2} ≺ L_{3}$

$≺$ — read "sits under": $L_{1}$ is at the bottom, $L_{3}$ on top.
The effective filesystem = overlay of all three: a file in $L_{3}$ hides any same-named file below (top sheet wins).
Each $L_{i}$ stores only its diff — that is why $L_{3}$ (flask) is a few MB, not the whole OS.

Step 3 — The content hash: why a layer knows if it changed

WHAT. We attach to every layer a digest — a fixed-length fingerprint computed from the layer's contents.

WHY. This is the single most important idea for understanding caching. Docker needs a cheap way to answer "is this layer the same as last time?" without comparing every byte. A hash does exactly that: same input bytes → same fingerprint; change one byte → totally different fingerprint.

PICTURE. Each sheet carries a red tag — its hash. Change nothing, the tag stays identical. We edit one file in the middle sheet, and its tag flips to a new value.

$digest (L_{i}) = H (the hash below it parent digest ∥ what this step added this layer’s diff)$

$H (\cdot)$ — a hash function: eats bytes, spits a fingerprint.
$∥$ — "glued together": the input is parent hash plus own diff.
The crucial consequence: because the parent digest is baked in, if a lower layer changes, its digest changes, which changes the input to the layer above, which changes that layer's digest — a chain reaction upward.

Keep that chain reaction in mind — it drives Step 5.

Step 4 — `docker run`: adding the one writable sheet

WHAT. We turn the (read-only) image into a live container by placing one thin writable sheet on top.

WHY. An image is frozen — you cannot write to it, and ten containers must be able to share it without stepping on each other. The fix: keep all shared layers read-only, and give each container its own private scratch sheet for anything it writes.

PICTURE. The red sheet at the very top is the writable container layer. The layers below are shared, untouched. Two containers from the same image are drawn: they share the identical read-only stack and each own a separate red top sheet.

$Container = read-only, shared [L_{1}, \dots, L_{n}] + writable, private W$

$W$ — the writable layer, one per running container.
Writes land in $W$ (copy-on-write: to edit a lower file, it's first copied up into $W$ ).
Sharing math: ten containers cost $10 \times ∣ W ∣$ (tiny) instead of $10 \times ∣ Image ∣$ (huge), because the $L_{i}$ are stored once.

Step 5 — The cache rule, and why order is everything

WHAT. During docker build, Docker walks the layers top-to-bottom of the Dockerfile and, for each, asks: "does a layer with this exact digest already exist?" If yes → reuse instantly. The moment one digest misses, every layer after it must be rebuilt.

WHY. This is the payoff of Step 3's chain reaction. We now prove the parent's advice — "order least-changing to most-changing" — is not a style preference but a direct consequence of hashing.

PICTURE (bad order). Top diagram: COPY . . (all your code) sits below pip install. You edit one line of code → the COPY diff changes → its digest flips → every layer above it, including pip install, is invalidated (red) and must re-run. A full dependency reinstall on every code edit.

Formally, let step $k$ be the first whose digest changed. Then:

$rebuilt layers = {L_{k}, L_{k + 1}, \dots, L_{n}}$

Everything below $k$ (unchanged inputs) → cache hit, free.
Everything from $k$ upward → cache miss, rebuilt, because each depends on the one below (Step 3).
Goal: make the frequently-changing instruction (COPY . .) have the largest $k$ — i.e. put it last — so a code edit rebuilds the fewest layers.

Step 6 — The fix, drawn: split the copy

WHAT. We copy the dependency manifest alone first, install, and copy the source code last.

COPY requirements.txt .              # rarely changes -> low, stays cached
RUN pip install -r requirements.txt  # stays cached with it
COPY . .                             # changes often -> pushed to the top

WHY. Now the layer whose digest flips on every code edit is the topmost one, so its chain reaction reaches nothing above — the expensive pip install stays cached.

PICTURE. Same edit-one-line-of-code scenario as Step 5, but only the top red sheet re-runs; pip install (green, cached) is untouched. Compare the amount of red against Step 5 — that red area is your build time.

$k_{bad} ≪ k_{good} \Rightarrow ∣ rebuilt_{good} ∣ ≪ ∣ rebuilt_{bad} ∣$

$k_{bad}$ — code copied early ⇒ small index ⇒ many layers above rebuilt.
$k_{good}$ — code copied last ⇒ largest index ⇒ (almost) nothing above to rebuild.
Same instructions, reordered — a free, dramatic speedup. Pure consequence of the hash chain.

Step 7 — Edge & degenerate cases (never leave the reader stranded)

WHAT. We sweep the corner cases the happy path skipped.

WHY. The contract: the reader must never hit a scenario we didn't show.

PICTURE. Four mini-panels, each with its outcome flagged in red.

Empty Dockerfile / only FROM scratch. scratch is a special empty base — zero layers below. The image is just your own layers; valid, used for tiny static binaries. Degenerate but legal: $n$ can start effectively from nothing.
A layer that only deletes files. Its diff is "remove X". But X still exists in the layer below! Total image size does not shrink — the delete is recorded as a whiteout on top. (Fix: don't add-then-delete across layers; do it within one RUN.)
Two images sharing a base. If both FROM python:3.12-slim, the base layers are stored once on disk and shared, even across different images. Sharing isn't just per-container (Step 4) — it's global by digest.
Change an early layer deliberately (e.g. bump the base image). Chain reaction rebuilds everything — expected and correct, because every later layer's assumptions changed. Slow, but rare, so we accept it.

The one-picture summary

This final figure compresses all seven steps: the Dockerfile on the left generating hashed read-only layers (the image), and docker run topping them with a private writable sheet (the container) — with the cache boundary that made ordering matter drawn in red.

Recall Feynman retelling — the whole walkthrough in plain words

Think of a Dockerfile as a stack of transparent instructions. Each instruction lays down one see-through sheet that only records what it changed — the first sheet is a whole starter operating system, the next adds Python, the next adds your libraries. Stack them and look down: you "see" the complete filesystem, but each sheet is stored tiny and separately.

Every sheet gets a fingerprint computed from its own contents plus the fingerprint of the sheet beneath it. So if you change a lower sheet, its fingerprint changes, which changes the one above's fingerprint, and so on all the way up — a domino effect.

When Docker rebuilds, it reuses any sheet whose fingerprint it has seen before and only remakes sheets from the first change upward. That's why you copy your rarely-changing dependency list early (so it stays cached) and your constantly-changing code last (so only the top sheet remakes). Reordering two lines can turn a 40-second build into 2 seconds — not magic, just fingerprints and dominoes.

To run the image, Docker slaps one writable sheet on top. That's the container. All the read-only sheets are shared by every container made from that image — cheap. But the writable sheet dies with the container, which is why data you care about must go on a volume instead. Recipe (image) is forever; the cake on the table (container) is temporary.

See also: the parent topic, Virtual Machines vs Containers, CI-CD Pipelines, Kubernetes, Microservices Architecture, Environment Variables and 12-Factor App.