NumPy broadcasting rules: Exercises — AI-ML deep dive

This page is a self-test ladder. Every problem builds on the parent's three rules — see the parent note if any symbol below feels unfamiliar. Before you peek, actually try each one. The solutions are hidden inside collapsible callouts.

Let us fix the vocabulary once, so nothing below is a mystery.

We read from the right because that is the axis that changes fastest in memory (row-major order), as the parent explained. Picture it below.

Notation reminder (earned before use):

shape = the tuple of sizes, e.g. (3, 4) means 3 rows, 4 columns.
np.newaxis (or None) = "insert a new axis of size 1 here". Turns a (3,) vector into a (3, 1) column.
A 0-D scalar has shape () — no axes at all, not a length-1 array.

Level 1 — Recognition

Recall Solution L1.1

Right-align, pad left with 1, walk each column.

(a) (4,3) vs (_,3) → dim -1: 3 vs 3 ✅ Rule 1; dim -2: 4 vs missing→1 ✅ Rule 3. Yes → (4, 3).
(b) (4,3) vs (_,4) → dim -1: 3 vs 4 ❌ no rule. No — ValueError. (The 4 lands on the wrong axis!)
(c) (5,1) vs (1,6) → dim -1: 1 vs 6 ✅ Rule 2; dim -2: 5 vs 1 ✅ Rule 2. Yes → (5, 6).
(d) (2,3) vs (2,4) → dim -1: 3 vs 4 ❌. No — ValueError.

Recall Solution L1.2

7 has shape () — zero axes. Rule 3 fills every missing axis with 1, giving an effective (1,1,1), and each 1 stretches by Rule 2. So a scalar broadcasts against any shape. Result: (2, 2, 2).

Level 2 — Application

Recall Solution L2.1

(3,4) vs (4,) → dim -1: 4 vs 4 ✅; dim -2: 3 vs missing→1 ✅. Works → (3, 4). Each of the 3 rows has the same 4-vector subtracted — exactly per-column centering. This is the core move in 2.1.02-Feature-normalization-standardization.

Recall Solution L2.2

(3,4) vs (3,) → dim -1: 4 vs 3 ❌. Fails — the (3,) aligns under the last axis 4. Fix: X - row_mean[:, np.newaxis] makes it (3,1). Then dim -1: 4 vs 1 ✅ stretch; dim -2: 3 vs 3 ✅. Result (3, 4).

Recall Solution L2.3

(3,1) vs (2,)→(1,2): dim -1 1 vs 2 stretch; dim -2 3 vs 1 stretch → (3,2). Entry [i,j] = a[i] + b[j]: $1 + 10 2 + 10 3 + 10 1 + 20 2 + 20 3 + 20 = 111213212223$

Level 3 — Analysis

Recall Solution L3.1

Pad B on the left: (2,1) → (1, 2, 1). Now vs (1,1,2):

dim -1: 2 vs 1 → Rule 2 → 2
dim -2: 1 vs 2 → Rule 2 → 2
dim -3: 1 vs 1 → Rule 1 → 1

Result (1, 2, 2). Notice the output rank (3) exceeds both inputs' printed shapes — the missing axis of B was silently filled.

Recall Solution L3.2

Result shape (1000, 1000) = 1,000,000 floats (this array is real and must exist). Broadcasting copies zero extra floats for the operands — the size-1 axes are stretched virtually (via zero-stride views), never materialised. See 1.2.04-Memory-efficient-Python-patterns.

Recall Solution L3.3

(P,Q): (8,1,6) vs (_,7,1) → -1: 6 vs 1 ✅; -2: 1 vs 7 ✅; -3: 8 vs miss→1 ✅. OK → (8,7,6).
(P,R): (8,1,6) vs (8,7,1) → -1: 6 vs 1 ✅; -2: 1 vs 7 ✅; -3: 8 vs 8 ✅. OK → (8,7,6).
(Q,R): (_,7,1) vs (8,7,1) → -1: 1 vs 1 ✅; -2: 7 vs 7 ✅; -3: miss→1 vs 8 ✅. OK → (8,7,1).

None fail! All three pairs broadcast. The trap was assuming different ranks must clash.

Level 4 — Synthesis

Recall Solution L4.1

Use X[:, None, :] → (5, 1, 2) and X[None, :, :] → (1, 5, 2). Subtract: dim -1 2 vs 2 ✅; dim -2 1 vs 5 stretch; dim -3 5 vs 1 stretch → (5, 5, 2) ✅. Squaring and summing over the last axis gives the squared-distance matrix used in ML.

Recall Solution L4.2

X - mu: (100,3) vs (3,) → -1 3v3 ✅, -2 100v miss→1 ✅ → (100,3).
.../ sigma: (100,3) vs (3,) → same reasoning → (100,3).

Both broadcasts land the length-3 vector on the feature axis. Directly powers 2.1.02-Feature-normalization-standardization.

Recall Solution L4.3

r = np.arange(1,5) shape (4,). Compute r[:, None] * r[None, :] → (4,1)*(1,4) → (4,4). Top-left 2×2: entries (1·1, 1·2; 2·1, 2·2) = $[1224]$

Level 5 — Mastery

Worked example L5.1 — Debug the shape error

This raises ValueError. Find the exact failing dimension and fix it with one reshape/newaxis.

gray = np.random.rand(32, 32)      # (32, 32) image
rgb_weights = np.array([0.3, 0.59, 0.11])  # (3,)
tinted = gray[..., None] * rgb_weights     # want (32, 32, 3)?

Actually this one works — instead debug the broken version below and say why:

tinted = gray * rgb_weights   # (32,32) * (3,)

Recall Solution L5.1

Broken line: (32,32) vs (3,) → dim -1 32 vs 3 ❌ → ValueError. Fix: add a trailing axis to gray: gray[..., None] → (32,32,1), then vs (3,)→(1,1,3): -1 1v3 stretch, -2 32v1 stretch, -3 32v1 stretch → (32,32,3) ✅. Each pixel becomes an RGB triple.

Recall Solution L5.2

First two: (3,1,5) vs (1,4,5) → -1 5v5 ✅, -2 1v4 stretch, -3 3v1 stretch → (3,4,5). Add (5,)→(1,1,5): -1 5v5 ✅, -2 4v1 stretch, -3 3v1 stretch → (3, 4, 5). No failure.

Recall Solution L5.3

(64,128) vs (128,) → -1 128v128 ✅, -2 64v miss→1 ✅ → (64,128). The same 128-length bias is added to every row (sample) — broadcasting supplies the repetition virtually, so we store only 128 numbers, not 64×128. This is the exact pattern in 3.2.04-Matrix-operations-in-neural-networks. For speed implications see 1.4.06-NumPy-performance-optimization.

Recall Quick self-check

Right-alignment pads the shorter shape on which side? ::: The left (leading axes), always with 1s. A size-1 axis stretches to any size — what's this called? ::: Rule 2, the singleton rule. How many real floats does broadcasting a (1,1000)+(1000,1) copy for its operands? ::: Zero — stretches are virtual (zero-stride views).