Foundations — Broadcasting — rules, why it works, gotchas

Before you can understand the parent note Broadcasting, you must be fluent in the words it throws around: shape, dimension, axis, size-1 axis, right-align, stride, stride-0, and the [:, None] notation. This page builds every one of them from absolute zero. Nothing below assumes you have seen NumPy before.

0. The rawest object: a strip of numbers

The single most important thing to unlearn: memory is always flat. A "2-D matrix" is a story we tell about a 1-D strip. The shape and strides are the story.

Look at the figure. The bottom is the true memory: six numbers in a row. The top is how we imagine it — a $2\times 3$ grid. Same six numbers; only the instruction card differs.

See NumPy Arrays — shape, dtype, strides for the full anatomy — here we only need enough to reach broadcasting.

1. Counting: 0,1,2,3 and why indexing starts at 0

Every position along a direction is labelled by a whole number, starting at 0. If a direction has 4 items, their labels are 0,1,2,3 — see the counting note. The last label is always size minus one, never size.

2. Dimension, axis, and shape — three words for the same picture

These three words get used interchangeably in the parent note. Pin them down:

3. The special hero: a size-1 axis

Broadcasting's whole magic lives on axes whose length is exactly 1. So we must see clearly what "a direction with only one slot" looks like.

The figure contrasts (3,1) (a column: 3 down, 1 across) with (1,4) (a row: 1 down, 4 across). The dashed arrows show the conceptual stretch: the single column value spreads rightward; the single row value spreads downward.

Note the two shapes below (they matter later): (3,) is a plain 1-D row of 3; (3,1) is a 2-D column. They contain the same three numbers but tell different stories — one has 1 axis, the other has 2.

4. Right-align: how two shapes are compared

The rules compare shapes by lining them up from the right. Here is why the right and not the left.

   A   ( 3 , 4 )        A   ( 3 , 1 )        A   ( 3 , 4 )
   B   ( 1 , 4 )        B   ( 1 , 4 )        B   ( 2 , 4 )
   ---------------      ---------------      ---------------
   ok:  3   4    ✓      ok:  3   4    ✓      3≠2 neither 1  ✗

The middle column of that last case is the only failure mode: two different numbers, neither of which is 1. See Reshaping and newaxis (None) indexing for how you fix a mismatch by inserting size-1 axes.

5. Stride, and the stride = 0 trick

Now the deepest word: stride. This is what makes broadcasting free.

Example: a (3,4) array of 8-byte numbers. To move one step across (axis 1, next column) you jump 8 bytes — the very next number. To move one step down (axis 0, next row) you must skip a whole row of 4 numbers = $4\times 8=32$ bytes. So its strides are (32, 8).

Now the punchline that powers the whole topic:

That single line — "stride 0" — is why the parent note can claim broadcasting costs no extra memory. There is no clever copying; there is one pointer that refuses to advance.

6. The [:, None] notation decoded

The parent note writes w[:, None] and calls it "adds a trailing size-1 dim." Symbol by symbol:

None here is the same idea as np.newaxis — see Reshaping and newaxis (None) indexing. This is the tool for fixing the orientation trap in Example 3 of the parent note: you insert a size-1 axis so your data lines up with the axis you actually meant.

7. Why broadcasting beats writing loops (the payoff)

Everything above exists so you can write A + v instead of a hand-written loop.

This is the same win described in Vectorization vs Python loops. Broadcasting is vectorization with automatic shape-matching. The honest alternative — physically duplicating the small array with np.tile and np.repeat (explicit replication) — gives the same answer but wastes the memory that stride-0 saves.

Prerequisite map

Equipment checklist

Cover the right side and test yourself. If any one fails, re-read that section before opening the parent note.