NumPy arrays and vectorized operations: Worked examples

What "broadcasting" even means (built from zero)

Before any example, we pin down the one rule everything below obeys. A NumPy array has a shape: a tuple like (3, 4) that just means "3 rows, 4 columns". Read it right-to-left: the last number is how wide each row is, the number before it is how many such rows, and so on outward.

We also need one more tiny piece of vocabulary that recurs below.

When you combine two arrays with +, *, etc., NumPy must decide how their grids line up. It uses one rule.

Why this rule and not something else? Because it is the least surprising way to let a small array act on a big one without copying memory. A size-1 axis literally has one value; stretching it means "reuse that value", which costs nothing. Two genuinely different sizes (like 4 and 5) have no honest way to line up, so NumPy refuses rather than guess.

Figure — NumPy arrays and vectorized operations

Look at the figure: the small row (chalk-blue) has shape (3,). Aligned under the big grid (4, 3), its single row is reused down all 4 rows (the pale-yellow ghost copies). No memory is duplicated — NumPy just reads the same three numbers four times.

The scenario matrix

Every problem on this topic lands in one of these cells. The worked examples below are tagged with the cell they cover.

#	Case class	Shapes / inputs	What can go wrong
A	Same-shape element-wise	`(n,) + (n,)`	trivial, but sets the baseline
B	Scalar broadcast	`(m,n) * scalar`	forgetting scalar = shape `()`
C	Row-vector broadcast	`(m,n) + (n,)`	wrong axis if you meant columns
D	Column-vector broadcast	`(m,n) + (m,1)`	need explicit `reshape`/`newaxis`
E	Outer / both stretch	`(m,1) + (1,n)` → `(m,n)`	surprise 2-D result
F	Higher-D stretch	`(4,1,3) + (5,3)` → `(4,5,3)`	mental alignment
G	Incompatible (fails)	`(3,4) + (5,)`	must recognise and fix
H	Zero / degenerate	shape `(0,)`, size-`0` axis	empty result, reductions
I	Feature normalisation (real-world)	`(100,3) - (3,)`	picking the right axis
J	Exam twist: `*` vs `@`	Hadamard vs matmul	wrong operator
K	In-place trap	`a += b` dtype clash	silent truncation

Example A — the baseline (cell A)

Forecast: guess both results before reading on.

Shapes match: both are (3,). Aligned from the right, the single column reads 3 vs 3 — equal, so compatible with no stretching. Why this step? Same-shape is the one case with no broadcasting; every slot pairs with the slot at the same index.
Add slot-by-slot: $c_{i} = a_{i} + b_{i} \Rightarrow [11, 22, 33]$ . Why? + on arrays is defined element-wise (parent's element-wise formula).
Multiply slot-by-slot: $c_{i} = a_{i} \times b_{i} \Rightarrow [10, 40, 90]$ . This is the Hadamard product, not a dot product. Why? * never means matrix/dot multiplication — that is reserved for @.

Verify: the sum of a+b is $11 + 22 + 33 = 66$ , which equals $sum (a) + sum (b) = 6 + 60$ . ✓

Example B — scalar broadcast (cell B)

Forecast: what shape comes out?

A scalar has shape () (empty tuple). Aligned against (2,2), it is padded to (1,1). Why this step? Understanding a scalar as a degenerate array explains why the rule still applies uniformly.
Both axes are size 1, so the single value 3 (or 100) stretches over all 4 slots. Why? A size-1 axis is reused for every slot, so the lone scalar reaches every cell of the grid at no memory cost — that is what makes 3 * M a whole-array operation, not a loop.
Result: 3*M = [[3,6],[9,12]], M+100 = [[101,102],[103,104]]. Same shape (2,2) out. Why? Broadcasting never shrinks the bigger array; the output shape is the elementwise maximum of the aligned axes, here (2,2).

Verify: total of 3*M = $3 + 6 + 9 + 12 = 30 = 3 \times (1 + 2 + 3 + 4) = 3 \times 10$ . ✓

Example C — row-vector broadcast (cell C)

Forecast: which axis gets stretched — rows or columns?

Align from the right: (4, 3) and (3,) → pad the second to (1, 3). Why this step? Padding on the left with 1 is the rule; it turns the flat vector into "1 row of 3".
Compare columns: last axis 3 vs 3 (equal ✓); first axis 4 vs 1 (one is 1, stretch ✓). Why? The size-1 first axis means "one row"; stretching reuses that row 4 times.
Result shape (4,3): each output row is G[i] + [100,200,300].

Look at the chalk-blue row sliding down: it is added identically to each of the 4 board rows. This is the checkerboard-column picture from the parent's recall callout.

Verify: if G = [[0,0,0],[1,1,1],[2,2,2],[3,3,3]], row 3 (0-indexed) is [3,3,3], so it becomes [103,203,303]; its sum is $103 + 203 + 303 = 609$ . ✓

Example D — column-vector broadcast (cell D)

Forecast: will G + col work as written? (Careful!)

Naive attempt fails: col has shape (4,). Align: (4,3) vs (1,4) → last axis 3 vs 4, neither is 1 → ValueError. Why this step? This is the classic trap: a length-4 vector wants to be a column, but NumPy reads flat vectors as rows.
Fix — make it a column: reshape to (4,1) via col[:, None] (the None/np.newaxis inserts a size-1 axis). Why? Now align (4,3) vs (4,1): first axis 4=4 ✓, last axis 3 vs 1 stretch ✓.
Result shape (4,3): row $i$ gets col[i] added to all 3 of its entries.

Verify: row 0 of G=[[0,0,0],...] becomes [10,10,10], sum $30 = 3 \times 10$ . ✓

Example E — both axes stretch, an "outer" pattern (cell E)

Forecast: what shape? (Trick: neither input is 2-D-looking yet the answer is a full grid.)

Align: (3,1) and (1,2). Why this step? Both arrays have a size-1 axis, so both get stretched — this is how you build a table from two vectors.
First axis (axis 0): 3 vs 1 → stretch to 3. Second axis (axis 1): 1 vs 2 → stretch to 2. Result (3,2).
Fill: entry $(i, j) = x_{i} + y_{j}$ . In plain words: the answer is a 3-row, 2-column table whose row $i$ , column $j$ cell is $x_{i} + y_{j}$ , giving rows [11, 21], [12, 22], [13, 23]:

111213212223

Look at the figure: the chalk-blue column x stretches sideways, the chalk-pink row y stretches downward, and where they cross (pale-yellow) each cell holds $x_{i} + y_{j}$ .

Verify: entry $(2, 1)$ (row index 2, column index 1) $= x_{2} + y_{1} = 3 + 20 = 23$ . ✓ Grand total $= 3 \cdot (y_{0} + y_{1}) + 2 \cdot (x_{0} + x_{1} + x_{2}) = 3 \cdot 30 + 2 \cdot 6 = 102$ . ✓

Example F — higher-dimensional stretch (cell F)

Forecast: write your guess (this is a real interview question).

Recall from the axis definition above: axis 0 is the leftmost slot of the shape, axis 1 the middle, axis 2 the rightmost.

Pad the shorter shape on the left: B becomes (1, 5, 3). Why this step? Both must have the same number of axes before comparing; missing left axes are treated as size 1.
Compare axis by axis (positions counted from the left): axis 0 is 4 vs 1→4; axis 1 is 1 vs 5→5; axis 2 is 3 vs 3→3. Why? Each axis is judged independently by the same equal-or-one rule; a size-1 axis in either array yields the other's size.
Result shape (4, 5, 3) — exactly the parent flashcard's answer.

Look at the figure: think of the result as 4 stacked sheets (axis 0), each sheet a 5×3 grid (axis 1 × axis 2). A supplies the same 1×3 strip repeated down each sheet; B supplies the same 5×3 face repeated across all 4 sheets.

Verify: the output has $4 \times 5 \times 3 = 60$ elements. ✓

Example G — the failure case, and how to read the error (cell G)

Forecast: which axis is the culprit?

Align: (3,4) and (5,) → pad to (1,5). Why this step? Always align first; the error always lives in one specific column.
Compare: last axis 4 vs 5 — different, and neither is 1 → broadcasting fails. NumPy says "operands could not be broadcast together with shapes (3,4) (5,)". Why? The rule stretches only size-1 axes; 4 and 5 are both genuine sizes, so there is no honest one-to-one pairing and NumPy refuses rather than silently guess.
Fix (if you meant a per-column add): the vector must have length 4, not 5, or shape (3,1) for a per-row add. There is no correct broadcast of a length-5 vector against a width-4 grid — the mismatch is real, not a syntax slip. Why? Choosing the right length (or inserting a size-1 axis) is the only way to make one aligned column satisfy the equal-or-one test; you cannot repair a 4-vs-5 clash by any reshape that keeps the 5.

Verify: 4 != 5 and min(4,5) != 1, so the rule's compatibility test is False. ✓

Example H — zero-size and degenerate inputs (cell H)

Forecast: empty does not mean "error" — guess each.

np.array([]).shape is (0,): a valid array with zero elements along its one axis. Why this step? Zero is a legal size; the array exists, it just holds nothing.
[] + []: align (0,) vs (0,) → 0=0 ✓ → result shape (0,), an empty array. No error. Why? Two equal axis sizes are compatible even when that size is 0; the elementwise loop simply runs zero times.
np.sum of empty is 0.0: the sum over no numbers is the additive identity. Why? Reductions return the identity element on empty input — sum→0, product→1.
np.array([[]]).shape is (1, 0): one row, zero columns. A degenerate 2-D grid. Why? The outer bracket gives one row (axis 0 = 1), the inner empty list gives zero columns (axis 1 = 0), so the shape faithfully records both.

Verify: np.sum([]) == 0 and shapes are (0,) and (1,0). ✓

Example I — real-world word problem: feature normalisation (cell I)

Forecast: should mean be over axis=0 or axis=1?

Compute per-column mean: mu = X.mean(axis=0) gives shape (3,). Why this step? axis=0 collapses the rows (the 100 samples), leaving one number per feature — exactly what "per-column" means.
Same for spread: sigma = X.std(axis=0), shape (3,). Why? Spread must be measured over the same axis as the mean so that each feature is scaled by its own variability, not another's.
Broadcast the subtraction: X - mu aligns (100,3) with (3,)→(1,3), stretching the single stats-row down all 100 rows (this is Example C's pattern). Why? The per-feature mean is one value per column; broadcasting reuses that row for every sample, so every sample's feature is centred by the correct column mean.
Divide element-wise: / sigma broadcasts identically. Result shape (100,3). Why? Dividing after subtracting rescales each centred feature to unit spread; the same (3,)→(1,3) stretch keeps each column matched to its own sigma.

Verify (concrete tiny case): take X = [[1,10],[3,30]]. Column means are [2,20], column stds (population) are [1,10]. Normalised = [[-1,-1],[1,1]]; each column now has mean 0. ✓ This same vectorised pattern feeds 2.2.04-Batch-gradient-descent.

Example J — exam twist: `*` vs `@` (cell J)

Forecast: which one keeps the off-diagonal 0s of B?

A * B is Hadamard (element-wise): multiply matching cells.

A * B = [1 \cdot 10 3 \cdot 0 2 \cdot 0 4 \cdot 10] = [100040]

Why this step? * never sums across a row; each output cell depends on exactly one cell of each input, so B's zeros wipe out the matching cells of A. 2. A @ B is matrix multiply: cell $(i, j)$ is the dot product of row $i$ of A with column $j$ of B, i.e. $(A B)_{ij} = \sum_{k} A_{ik} B_{k j}$ .

A @ B = [1 \cdot 10 + 2 \cdot 0 3 \cdot 10 + 4 \cdot 0 1 \cdot 0 + 2 \cdot 10 3 \cdot 0 + 4 \cdot 10] = [10302040]

Why? @ sums a full row-times-column dot product per cell (the parent's BLAS formula), so each output cell mixes an entire row of A with an entire column of B. 3. Reading the result: here B is 10·I (10 times the identity), so multiplying by it on the right just scales every entry of A by 10 — that is why A @ B = 10·A = [[10,20],[30,40]]. But A * B is not 10·A: it keeps only the diagonal ([10,40]) and zeroes the off-diagonals, because * copies B's zeros straight through. Same two inputs, completely different outputs. Why? The two operators answer different questions — * asks "combine cells at the same position", @ asks "combine each row with each column" — so identical inputs legitimately give different grids.

Verify: (A*B) = [[10,0],[0,40]]; (A@B) = [[10,20],[30,40]] = 10*A. ✓ Deeper matmul work: 2.1.03-Matrix-operations-for-ML and 3.1.02-Tensor-operations-in-PyTorch.

Example K — the in-place dtype trap (cell K)

Forecast: do both give [1.5, 2.5, 3.5]?

a = a + 0.5 creates a NEW array: NumPy promotes to float64, giving [1.5, 2.5, 3.5]. Why this step? A fresh array is free to choose the wider dtype that fits the result, so no precision is lost.
a += 0.5 writes IN PLACE: it must keep a's existing int64 buffer. The float result is cast back to int, truncating toward zero → [1, 2, 3] (older NumPy) or raises a UFuncTypeError (newer NumPy). Why? In-place ops reuse the same memory (the parent's performance tip #2), and an int64 buffer physically cannot hold 1.5, so the fractional part is dropped or the op is refused.
Rule of thumb: use in-place +=/*= only when the two dtypes are already compatible (both int, or both float). Why? Matching dtypes guarantee the result fits the reused buffer, so you gain the speed of in-place without the silent truncation.

Verify: int64 truncation of [1.5,2.5,3.5] toward zero is [1,2,3]. ✓

Recall Quick self-test

Shape of (6,1) + (1,4)? ::: (6, 4) — both size-1 axes stretch. Does (3,4) + (4,) work? ::: Yes — pads to (1,4), last axis 4=4, first 3-vs-1 stretch. Does (3,4) + (3,) work? ::: No — last axis 4 vs 3, neither is 1 → fails. Reshape to (3,1). np.sum(np.array([])) = ? ::: 0.0, the additive identity. A * B vs A @ B — which sums across a row? ::: @ (matrix multiply); * is element-wise. In shape (4,5,3), which number is axis 1? ::: 5 — axes count from 0 on the left. a += 0.5 on an int64 array gives? ::: [1,2,3] (truncated) or a UFuncTypeError — never [1.5,2.5,3.5].

Connections

Parent: NumPy arrays and vectorized operations — the what; this page is the do every case.
1.4.01-Python-fundamentals — scalars, tuples (shapes are tuples).
1.4.02-List-comprehensions-and-generators — the slow loops broadcasting replaces.
1.4.05-Pandas-DataFrames — column-wise ops (Example I) reappear as DataFrame ops.
2.1.03-Matrix-operations-for-ML — @ and matmul rules (Example J).
2.2.04-Batch-gradient-descent — per-column normalisation and vectorised gradients.
3.1.02-Tensor-operations-in-PyTorch — identical broadcasting rules on tensors.