Vectorization — avoiding Python loops, speed comparison: Question bank

This bank drills the boundary cases and misconceptions from the parent note: Vectorization. For the machinery behind each answer see NumPy Arrays vs Python Lists, Broadcasting Rules, Universal Functions (ufuncs), Big-O Complexity, Numba and Cython, and Memory Layout and Cache.

Notation used in this bank

Several traps below reuse the parent note's time model, so let's pin the symbols down before any of them appear. The Greek letter $τ$ (tau) just means "time per one element" — a per-element cost measured in nanoseconds.

So $τ_{p y}$ and $τ_{v ec}$ are just the two concrete values that the generic $τ_{per-element}$ takes in the Python-loop case versus the vectorized case. Every formula below uses exactly these.

Look at the picture below before reading on — it shows why this ratio starts small (setup dominates) and climbs to a ceiling as $N$ grows.

Figure — Vectorization — avoiding Python loops, speed comparison

True or false — justify

Vectorization makes the algorithm asymptotically faster (better Big-O).

False. It attacks the constant factor per element, not the growth order — adding two arrays is

O (N)

whether looped or vectorized; vectorization just shrinks

τ_{per-element}

. See Big-O Complexity.

A vectorized operation is always faster than the equivalent Python loop.

False. For tiny

N

the fixed C-call setup

c_{setup}

dominates the denominator of the speedup ratio, so a scalar or short loop can win.

np.vectorize(f) compiles your Python function f into fast machine code.

False. It is a convenience wrapper that loops in Python and calls f per element; the speed is barely above a hand-written loop. Real compilation needs ufuncs or Numba and Cython.

Iterating with for x in numpy_array is at least as fast as iterating a Python list.

False. It is often slower: each raw float64 must be boxed into a np.float64 object every step, adding overhead a list never pays.

a[a < 0] = 0 and the loop version for i: if a[i]<0: a[i]=0 give the same result.

True. Both implement ReLU-clamp; the mask version just runs the comparison and the write inside compiled C instead of the Python interpreter.

Building an array by calling np.append in a loop is

O (N)

total, like list.append.

False. np.append reallocates and copies the whole array every call, giving

O (N^{2})

— worse than a list. Collect in a list then np.array once.

When each element needs heavy work (say sin then exp), the absolute time saved by vectorizing is larger than for plain addition.

True. Heavier per-element work means more Python overhead (boxing plus per-op dispatch) is eliminated in absolute terms, though whether the relative speedup grows depends on how memory-bound the op is.

At very large

N

, doubling the CPU clock speed will roughly halve a vectorized array-add.

False. Large vectorized ops are usually memory-bandwidth bound: the CPU waits on data from RAM, so faster arithmetic barely helps. See Memory Layout and Cache.

Spot the error

out = np.array([]); for x in data: out = np.append(out, x*x) — what's wrong?

The repeated np.append copies the growing array each iteration (

O (N^{2})

) and you still loop in Python. Use out = data**2.

M = [[u+v for v in col] for u in row] was "vectorized" to M = row + col. Why is that wrong?

Plain row + col does an element-wise sum of equal-length vectors, not the outer/pairwise matrix. You need broadcasting: row[:,None] + col[None,:]. See Broadcasting Rules and the figure below.

Someone writes total = 0; for x in arr: total += x and calls it fast because arr is a NumPy array. Error?

The loop still runs in Python and unboxes each element; use arr.sum(), which accumulates in C and may use SIMD.

result = np.vectorize(my_slow_python_fn)(big_array) is submitted as an optimization. Error?

np.vectorize doesn't compile anything — it's a Python-level loop, so my_slow_python_fn runs once per element just as slowly. Rewrite with ufuncs or Numba and Cython.

To "speed up", a coder replaces a + b with np.array([a[i]+b[i] for i in range(len(a))]). Error?

This reintroduces the Python loop plus a list-build plus a conversion — strictly slower than the already-vectorized a + b.

data.append(x) is used on a NumPy array inside a loop. Error?

NumPy ndarray has no .append method (that's a list method); the correct free function np.append also copies. Pre-allocate np.empty(N) and fill by index instead.

Why questions

Why does the speedup ratio

\frac{N τ _{p y}}{c _{setup} + N τ _{v ec}}

rise as

N

grows?

N

grows the constant

c_{setup}

becomes negligible next to

N τ_{v ec}

, so the ratio approaches the pure per-element ratio

τ_{p y} / τ_{v ec}

Why is a raw float64 in a NumPy array cheaper to add than a Python int?

The NumPy value is a packed machine number needing no type check, dispatch, or object allocation — the C loop does one CPU instruction; the Python object needs dozens.

Why can pre-allocating np.empty(N) and filling it beat growing an array?

It allocates the final memory block once, so each write is

O (1)

with no copying — total

O (N)

instead of the

O (N^{2})

of repeated reallocation.

Why does the assignment a[a < 0] = 0 remove the branch from Python?

The comparison a < 0 builds an entire boolean array in compiled C, and the masked assignment writes the matching elements in C too — so the per-element if decision never returns to the Python interpreter.

Why does vectorization rely on arrays being contiguous in memory?

A tight contiguous block lets the C loop stride linearly and lets the CPU prefetch cache lines; scattered objects (like a list of boxed numbers) defeat that. See Memory Layout and Cache and the figure below.

Why might numba beat pure NumPy for a complicated per-element formula?

NumPy runs each ufunc as a separate C pass (extra memory traffic between temporaries), while numba can fuse the whole formula into one compiled loop, cutting memory passes. See Numba and Cython.

Edge cases

For

N = 3

, does vectorizing a + b help?

Almost never — the fixed C-call setup

c_{setup}

swamps the tiny

N τ_{v ec}

work, so scalar arithmetic or a short loop can be as fast or faster.

What happens to the speedup when the vectorized op becomes memory-bandwidth bound?

The measured speedup plateaus below the theoretical

τ_{p y} / τ_{v ec}

: the CPU idles waiting for RAM, so removing arithmetic overhead stops helping.

Two arrays of shapes (4,) and (3,) are added — what occurs?

A broadcast error: the trailing dimensions are unequal and neither is 1, so NumPy raises a ValueError rather than looping silently. See Broadcasting Rules.

Is a[a < 0] = 0 still valid if no element is negative?

Yes. The mask is all-False, so zero elements are written; the operation is a harmless no-op, not an error.

What does a + b return when both a and b are empty arrays (length 0)?

An empty array, no error: the model gives

T = c_{setup} + 0 \cdot τ_{v ec} = c_{setup}

, so you pay only the fixed C-call cost with zero useful work — pure overhead.

Does an empty array trigger memory-bound behavior, and can you broadcast (0,) with (3,)?

There is no memory-bound effect (no data to move), and broadcasting (0,) with (3,) fails with a ValueError since

0 \neq = 3

and neither axis is 1; only (0,) with (1,) or another (0,) broadcasts.

Does vectorizing help when each iteration depends on the previous result (e.g. x[i] = x[i-1]*r)?

Not directly — such recurrences aren't element-independent, so a plain ufunc can't express them; use np.cumprod/cumsum-style scans or Numba and Cython for a compiled loop.

If two vectors already have length 1, is broadcasting them meaningful?

Yes, but trivial — a length-1 axis stretches to match the other, so a[:,None]+b[None,:] still produces a valid (1×1 or larger) grid with no error.

Connections

Vectorization — avoiding Python loops, speed comparison — the parent concepts these traps test.
NumPy Arrays vs Python Lists — why boxed objects are the slow part.
Broadcasting Rules — shape-mismatch edge cases above.
Big-O Complexity — constant-factor vs growth-order distinction.
Numba and Cython — when true compilation beats plain NumPy.
Memory Layout and Cache — the bandwidth ceiling on speedups.