Visual walkthrough — NumPy — ndarray structure, dtype, shape, strides

We build one idea: the offset formula

$\text{byte\_offset}(i,j)=i\cdot {\text{stride}}_0+j\cdot {\text{stride}}_1$

and we don't allow ourselves to write it until every symbol has been drawn.

---

Step 1 — Memory is a single numbered shelf

WHAT. Before any array, there is just computer memory: a long line of little storage cells, side by side, each with an address — a whole-number label $0,1,2,3,\dots$. Think of a shelf of lockers, each locker holding exactly one byte (a byte = the smallest chunk memory gives out, 8 bits).

WHY start here. Everything NumPy does is a trick of interpretation on this shelf. If you don't see the shelf, "strides" sounds like magic. Once you see it, strides are just "how many lockers do I jump?"

PICTURE. The blue lockers below are memory. The number under each is its address in bytes. Nothing is 2-dimensional yet — it's a single row.

---

Step 2 — A dtype groups bytes into one element

WHAT. Numbers don't fit in a single byte. An int64 (a 64-bit integer) needs $64/8=8$ bytes. So we glue 8 lockers together into one "element". The dtype is the rule that says how many bytes make one element and how to read those bytes as a number.

WHY this tool and not something else. We need a fixed group size. If elements could be different sizes (like Python lists, where each item can be anything), we'd have to store a pointer and look up each one — scattered, slow. A fixed dtype means every element is the same width, so "jump to the next element" is always the same distance. That constant distance is the whole reason strides work.

PICTURE. Same shelf as Step 1, now bracketed into groups of 8 bytes. Each orange bracket = one int64 element. Notice element $0$ occupies addresses $0-7$, element $1$ occupies $8-15$, and so on.

$\underset{\text{itemsize}}{\underbrace{s}}=8\text{bytes per element}$

• $s$ (the itemsize) is the width of one element in bytes. Here $s=8$. This single number appears in every stride formula later — remember it.

---

Step 3 — shape pretends the line is a grid

WHAT. We now decide to view our flat elements as a rectangle. Say we have $12$ elements and we choose to see them as 3 rows by 4 columns. That choice is the shape $=(3,4)$.

WHY. The bytes never move. shape is a promise: "read the flat elements in chunks of 4, stacking each chunk as a new row." We read row by row — fill row 0 left-to-right, then row 1, then row 2. This is called row-major or C order (see Row-major vs column-major (C vs Fortran order)).

PICTURE. Top: the flat line of 12 elements with element-index labels $0\dots 11$. Bottom: the same elements folded into a $(3,4)$ grid. The colored arrow shows the reading path snaking row by row. Element $9$ is highlighted so you can track it into the grid.

$\text{shape}=(\underset{=3}{\underbrace{R}},\underset{=4}{\underbrace{C}}),\text{total elements}=R\cdot C=12$

• $R$ = number of rows (size along axis 0, the vertical/down direction).
• $C$ = number of columns (size along axis 1, the horizontal/across direction).
• $R\cdot C$ must equal how many elements we actually have — otherwise the grid wouldn't be full.

---

Step 4 — The flat-index formula: how far into the line is A[i, j]?

WHAT. Given a row $i$ and column $j$, which element number (position in the flat line) is it? Because we filled row by row, to reach row $i$ we must skip $i$ whole rows, each holding $C$ elements — that's $i\cdot C$ elements. Then move $j$ steps into the current row.

WHY count this way. This is just the reading path from Step 3, turned into arithmetic. Every full row you pass costs you $C$ elements; every column you move costs you $1$.

PICTURE. The $(3,4)$ grid with element $A[2,1]$ circled. Green arrow: drop past 2 full rows ($2\times 4=8$ elements). Orange arrow: step 1 across. Total $=9$.

$\text{flat\_index}(i,j)=\underset{\text{skip }i\text{ full rows}}{\underbrace{i\cdot C}}+\underset{\text{move within row}}{\underbrace{j}}$

• $i\cdot C$ — each skipped row is $C$ elements long, and we skip $i$ of them.
• $j$ — after landing on row $i$, step $j$ elements sideways.

Check for $A[2,1]$ with $C=4$: $\text{flat\_index}=2\cdot 4+1=9$. ✓

---

Step 5 — Convert element-steps into byte-steps (strides are born)

WHAT. The CPU addresses memory in bytes, not elements. So multiply the flat index by the itemsize $s$ to get the actual byte offset.

WHY. Step 4 counted in elements; Step 2 told us each element is $s$ bytes wide. Multiplying stitches the two pictures together. This multiplication is exactly where strides appear.

PICTURE. The flat line drawn twice: top ruler in elements ($0,1,2,\dots$), bottom ruler in bytes ($0,8,16,\dots$). A red bracket shows one element-step $=$ 8 byte-steps.

Take the flat-index formula and multiply the whole thing by $s$:

$\text{byte\_offset}(i,j)=(i\cdot C+j)\cdot s=i\underset{{\text{stride}}_0}{\underbrace{(C\cdot s)}}+j\underset{{\text{stride}}_1}{\underbrace{(s)}}$

• $C\cdot s$ — moving down one row ($i\to i+1$) jumps past $C$ elements $=C\cdot s$ bytes. Call this ${\text{stride}}_0$.
• $s$ — moving across one column ($j\to j+1$) jumps 1 element $=s$ bytes. Call this ${\text{stride}}_1$.

---

Step 6 — Read the offset formula as a picture

WHAT. Let's see the two strides as two different jump-lengths on the shelf.

WHY. The formula $i\cdot {\text{stride}}_0+j\cdot {\text{stride}}_1$ says: take $i$ big jumps of length ${\text{stride}}_0$, then $j$ small jumps of length ${\text{stride}}_1$. That's it — that's all indexing is.

PICTURE. The flat byte-shelf. From the start we take $i=2$ big green jumps of $32$ bytes each (lands at byte $64$), then $j=1$ small orange jump of $8$ bytes (lands at byte $72$). Byte $72=$ element $72/8=9$.

$\text{byte\_offset}(2,1)=\underset{64}{\underbrace{2\cdot 32}}+\underset{8}{\underbrace{1\cdot 8}}=72\Rightarrow \text{element }\frac{72}{8}=9$

• $2\cdot 32$ — two down-steps, each a full row of 32 bytes.
• $1\cdot 8$ — one across-step of 8 bytes.
• Divide the final byte offset by $s$ to recover the element index whenever you want to check.

---

Step 7 — Degenerate & edge cases (the formula must never surprise you)

WHAT. Let's push the same formula into corners and confirm nothing breaks.

WHY. The contract: the reader must never meet a case we didn't show. Three corners matter — the top-left origin, a 1-D array, and the transpose where strides swap.

PICTURE. Three mini-panels sharing the byte-shelf idea:

1. Origin $A[0,0]$: $0\cdot 32+0\cdot 8=0$ → the very first byte. Zero jumps.
2. 1-D array shape $(12,)$: only one axis, one stride $=s=8$. Element $k$ sits at byte $8k$. Simple ruler.
3. Transpose A.T: shape becomes $(4,3)$ and the strides swap to $(8,32)$ — no byte moves. Now stepping down (axis 0) is the small 8-byte jump.

For the transpose, plug into the same formula. Element A.T[1, 2] with strides $(8,32)$:

$\text{byte\_offset}=1\cdot \underset{{\text{stride}}_0}{\underbrace{8}}+2\cdot \underset{{\text{stride}}_1}{\underbrace{32}}=8+64=72\Rightarrow \text{element }9$

• Same byte $72$, same value $9$ — because A.T[1,2] and A[2,1] are literally the same locker, just described with swapped strides. Transpose is a view: it shares memory.

---

The one-picture summary

WHAT. One figure ties the three metadata numbers to one journey across the shelf: dtype sets the group width $s$, shape sets $(R,C)$, and strides $=(C\cdot s,s)$ turn any $(i,j)$ into a byte address via $i\cdot {\text{stride}}_0+j\cdot {\text{stride}}_1$.

Recall Feynman retelling — the whole walkthrough in plain words

Start with a long shelf of lockers, each numbered by its byte address (Step 1). Glue lockers into groups of 8 so each group holds one full number — that gluing rule is the dtype (Step 2). Decide to pretend the row of groups is a 3×4 grid, reading left-to-right, top-to-bottom — that pretending is the shape (Step 3). To find "row 2, column 1", count: skip 2 whole rows of 4 elements, then 1 more — element number 9 (Step 4). But lockers are addressed in bytes, so multiply by 8 to get byte 72; the "how-many-bytes-per-step" numbers, 32 for down and 8 for across, are the strides (Steps 5–6). Test the corners: the origin is byte 0, a plain 1-D array has a single stride of 8, and flipping the grid sideways just swaps the two stride numbers without moving a single locker (Step 7). Offset = sum of (index × stride). That one sentence is NumPy indexing.

---

Related builds

• NumPy — views vs copies & np.shares_memory — why the transpose in Step 7 shares memory.
• Row-major vs column-major (C vs Fortran order) — the reading order chosen in Step 3.
• NumPy — broadcasting — the trick of setting a stride to 0 to "stretch" an axis.
• CPU cache & memory locality — why the last-axis-adjacent layout is fast.
• Python lists vs arrays — the scattered-pointer world we escaped in Step 2.
• NumPy — vectorization & performance — the payoff of one contiguous block.