Vectorization — avoiding Python loops, speed comparison

5.4.5 · Coding › Scientific Computing (Python)

WHY matter karta hai ye?

WHY slow hota hai Python loop? Python loop ki har iteration mein ye sab karna padta hai:

Agla object fetch karo (ek boxed PyObject, raw number nahi).
Runtime pe uska type check karo (Python dynamically typed hai).
Sahi __add__/__mul__ method ko dispatch karo.
Result ke liye ek naya Python object allocate karo.

Sirf do numbers add karne ke liye dozens of machine instructions lagte hain. Ek NumPy array raw float64 values ko memory mein tightly packed store karta hai, to C loop bas result[i] = a[i] + b[i] karta hai — ek CPU instruction, koi boxing nahi, koi type checks nahi.

WHAT hai speed model? (Scratch se derivation)

Chalo "100×" memorize karne ki jagah expected speedup derive karte hain.

$N$ elements pe ek operation ka total time:

$T = c_{setup} + N \cdot c_{per-element}$

Ek Python loop ke liye: $c_{per-element}^{py}$ bada hota hai (boxing, type check, dispatch). Ise $τ_{py} \approx 50 - 200$ ns kaho.
Ek vectorized op ke liye: $c_{per-element}^{vec}$ bas ek raw CPU op hai. Ise $τ_{vec} \approx 0.5 - 2$ ns kaho. Ek fixed C-call setup $c_{setup}$ hota hai.

Figure — Vectorization — avoiding Python loops, speed comparison

HOW karte hain vectorize? (Patterns)

Worked example 1. Element-wise arithmetic

# SLOW
out = []
for x in data:
    out.append(x*x + 1)
# FAST
out = data**2 + 1     # data is a np.ndarray

Ye step kyun? data**2 + 1 do compiled C loops trigger karta hai (square, add). Koi Python-level iteration nahi.

Worked example 2. Reductions

# SLOW
total = 0
for x in data: total += x
# FAST
total = data.sum()   # or np.sum(data)

Kyun? .sum() accumulation C mein run karta hai aur SIMD instructions bhi use kar sakta hai.

Worked example 3. Masks se conditional logic (loop mein

if replace karo)

# SLOW
for i in range(len(a)):
    if a[i] < 0: a[i] = 0     # ReLU
# FAST
a = np.where(a < 0, 0, a)     # or: a[a < 0] = 0

Kyun? a < 0 C mein ek boolean mask array banata hai; np.where C mein per-element select karta hai. Branch Python se hat jaata hai.

Worked example 4. Broadcasting (nested loops se bachao)

# pairwise sums of two vectors, SLOW double loop
M = [[u + v for v in col] for u in row]
# FAST
M = row[:, None] + col[None, :]   # shape (len(row), len(col))

Kyun? row[:,None] ek column banata hai, col[None,:] ek row; NumPy unhe C ke andar ek full matrix mein broadcast karta hai — bilkul bhi Python loops nahi.

Common mistakes (Steel-manned)

Forecast-then-Verify

Recall Feynman: 12-saal ke bacche ko explain karo

Imagine karo tumhe 1,000,000 envelopes stamp karne hain. Tarika 1 (Python loop): har envelope ke liye tum almari pe jaao, stamp uthao, instructions padho, stamp lagao, stamp wapas rakho, ek note likho. Ek million baar repeat karo — bahut thaka dene wala! Tarika 2 (vectorization): tum ek fast robot ko kaho "in sabko ek hi tarah se stamp kar do," aur wo poori pile ek smooth motion mein karta hai bina har baar instructions re-read kiye. Same result, lekin robot boring setup pe ek million baar time waste nahi karta. Isliye vectorized code itna faster hota hai — ye repetitive thinking ek baar karta hai, har item ke liye ek baar nahi.

Flashcards

Python for loop numbers pe NumPy se slow kyun hota hai?

Har iteration objects box karta hai, runtime type-checking karta hai, method dispatch karta hai, aur new-object allocation karta hai — har element ke liye ek CPU op ki jagah dozens of instructions.

"Vectorization" ka matlab kya hai?

Ek operation ko poore array pe ek saath express karna taaki looping compiled C mein run ho, Python interpreter mein nahi.

Array op ke liye time model do.

T = c_{setup} + N τ_{per-element}

; speedup

\to τ_{p y} / τ_{v ec}

large

N

ke liye.

Bahut chhote arrays ke liye, kya vectorization hamesha faster hai?

Nahi — fixed C-call setup

c_{setup}

dominate karta hai, to scalar/loop faster ho sakta hai.

Loop mein np.append kyun bura hai?

Ye har call pe poora array copy karta hai →

O (N^{2})

. List banao phir np.array, ya np.empty pre-allocate karo.

Kya np.vectorize(f) f ko fast banata hai?

Nahi — ye essentially ek Python-level loop hai; real speed ke liye NumPy ufuncs ya numba use karo.

Per-element if (jaise ReLU) ko vectorize kaise karte hain?

Boolean mask use karo: a[a<0]=0 ya np.where(a<0,0,a).

Broadcasting kis kaam aati hai?

Different shapes ke arrays combine karne ke liye (jaise row[:,None]+col[None,:]) nested Python loops se bachne ke liye.

for x in np_array iterate karna list se slower kyun ho sakta hai?

Har element ek np.float64 object mein unbox hota hai, normal Python loop se zyada overhead add karta hai.

Large N pe vectorized speed ko kya limit karta hai?

Memory bandwidth — CPU arithmetic ka wait nahi karta, data ka wait karta hai.

Connections

NumPy Arrays vs Python Lists — contiguous typed memory isliye vectorization kaam karta hai.
Broadcasting Rules — multi-dimensional combinations vectorize karna.
Universal Functions (ufuncs) — compiled C kernels jo aap call karte ho.
Big-O Complexity — isliye np.append-in-loop $O (N^{2})$ hai.
Numba and Cython — jab vectorization bhi kaafi nahi hota.
Memory Layout and Cache — vectorized ops pe bandwidth limits.