Jupyter notebooks workflow: Visual walkthrough

Parent topic: Jupyter notebooks workflow.

Before we start, one word we will use constantly:

We will draw this table as a box of name→value slots and watch it change. That single picture is the whole derivation.

Step 1 — One kernel, one shared memory box

WHAT. When you open a notebook, one Python process (the kernel) starts. It creates one empty namespace table. Every cell on the page talks to this same table.

WHY. This is the root cause of everything else. If each cell had its own private memory, order would not matter — but they don't. They share. Sharing is the feature (results persist between cells) and the trap (a cell can depend on memory a later cell created).

PICTURE. Below: three code cells on the left, but only one memory box on the right, with arrows from every cell pointing at it.

$namespace = {} (starts empty)$

$namespace$ — the lookup table living inside the kernel.
${}$ — the empty-dictionary symbol: no names defined yet.

Step 2 — Running a cell writes into the box

WHAT. Run a cell containing x = 5. The kernel evaluates the right side ( $5$ ) and stores it under the key "x".

WHY. We need to see writing before we can see reading. This is the only way a name ever enters memory: some cell must run and assign it. Position on the page does nothing; only running writes.

PICTURE. The arrow drops the value 5 into the slot labelled x. The slot was empty before, filled after.

$namespace ["x"] = 5$

$namespace ["x"]$ — the slot found by looking up the key "x".
$= 5$ — we place the value $5$ into that slot.

Step 3 — A later cell reads from the box

WHAT. Now run a second cell: y = x * 2. To compute the right side, the kernel must read "x" from the table, find $5$ , multiply, and write the result under "y".

WHY. This is the moment a dependency is born. Cell 2 cannot stand alone — it needs the slot x to already be filled. Who filled it? Whatever cell ran x = 5 earlier in time. Not earlier on the page — earlier in time.

PICTURE. A read-arrow leaves the x slot, feeds the multiplication, and a write-arrow drops $10$ into the new y slot.

$y = read = 5 namespace ["x"] \times 2 = 10$

$namespace ["x"]$ — a read: look up "x", get $5$ .
$\times 2$ — the arithmetic the cell asked for.
$= 10$ — the value written into the "y" slot.

Step 4 — The execution counter `In [n]` records time, not position

WHAT. Every time you run any cell, the kernel bumps a counter and stamps that number next to the cell as In [n].

WHY. The page has two different orderings and they can disagree:

Position order — top to bottom, how the cells are laid out.
Execution order — the order in which you actually pressed run, in time.

The counter is the only thing on the page that reveals execution order. Everything else (the code, the saved outputs) hides it.

PICTURE. Two cells: the top cell stamped In [12], the bottom cell stamped In [5]. The bottom one ran first even though it sits lower. The red arrow shows time flowing opposite to position.

$n_{new} = (largest n so far) + 1$

$n_{new}$ — the number stamped on the cell you just ran.
$largest n so far$ — the highest count reached previously.
$+ 1$ — one run, one tick. Re-running the same cell gives it a new, higher number.

Recall

If cell A shows In [12] and cell B (below it) shows In [5], which ran first? ::: Cell B — smaller counter means earlier in time, regardless of page position.

Step 5 — The trap: a cell that reads a slot written below it

WHAT. Suppose Cell 2 (upper on page) contains print(z), and Cell 4 (lower on page) contains z = 99. You happened to run Cell 4 first, then Cell 2. It worked for you.

WHY. When you ran Cell 4, the slot z got filled ( $99$ ). Then Cell 2 read it and found a value. The box didn't care that Cell 4 sits lower on the page — only that it ran earlier in time (Step 4).

PICTURE. Page position: Cell 2 above Cell 4. Execution arrows (red) run Cell 4 → Cell 2, backwards. The z slot is filled by the lower cell and read by the upper cell.

$Cell 2 reads namespace ["z"] \Leftarrow Cell 4 wrote it (earlier in time)$

$\Leftarrow$ — points to the cell that actually supplied the value.
The whole illusion lives here: page says "Cell 2 first", time says "Cell 4 first".

Step 6 — What your collaborator sees (the failure)

WHAT. You save and share the .ipynb. The file stores the code and the last outputs, but not the execution order. Your collaborator does the honest thing: run top to bottom.

WHY. Top-to-bottom means Cell 2 (print(z)) runs before Cell 4 (z = 99). At that instant the slot z is still empty. Reading an empty slot is not allowed.

PICTURE. Fresh empty box. Cell 2 runs first, reaches for the z slot, finds nothing → red NameError explosion. Cell 4 (which would have fixed it) never got a chance.

$read namespace ["z"] \to key missing \to NameError$

$key missing$ — "z" is not a key in the fresh table yet.
$NameError$ — Python's way of saying "you asked me to read a name I never stored."

Step 7 — The fix as a picture: Restart & Run All

WHAT. "Restart" throws the entire memory box away — a brand new empty table. "Run All" then executes cells strictly top to bottom.

WHY. This forces execution order to equal page position. If any cell reads a slot not yet written by a cell above it, it fails for you, now — exactly the failure your collaborator would hit. You catch it before sharing.

PICTURE. Left: old cluttered box → trash. Right: fresh empty box, arrows running strictly downward $1 \to 2 \to 3 \to 4$ , counters coming out clean as In [1], In [2], In [3], In [4].

$execution order = page order ⟹ reproducible$

Left side — execution order.
Right side — top-to-bottom page order.
$=$ — when these two match, anyone re-running from the top gets your result. This is the definition of a reproducible notebook.

The one-picture summary

Everything compressed: one shared box, cells that write slots and cells that read slots, an out-of-order run that secretly works, the same page failing top-to-bottom for a stranger, and the "Restart & Run All" broom that snaps execution order back onto page order.

Recall

Feynman retelling — say it like you'd explain to a friend. ::: There's one shared box of labelled slots. Running a cell either drops a value into a slot or reads one out. The catch: reading needs the slot already filled, and slots get filled by whatever cell you ran first in time — which need not be the cell sitting highest on the page. You might run cells out of order, filling a slot from a low cell that a high cell then reads, and it works — for you. But the saved file only remembers the code and last outputs, not the order you clicked. So a friend running top-to-bottom hits an empty slot and gets a NameError. The cure is to throw the box away (Restart) and run strictly downward (Run All): now the order you execute equals the order on the page, so if it survives for you, it survives for everyone. The little In [n] numbers are the only honest clock on the page — smaller number, earlier in time.

See also: 1.4.10-NumPy-arrays-basics, 1.5.01-Pandas-DataFrames, 2.3.01-Exploratory-data-analysis, 4.1.05-Debugging-strategies, 1.4.08-Virtual-environments, 1.6.01-Matplotlib-basics.