Visual walkthrough — Vectorization — avoiding Python loops, speed comparison

We will define every letter before we use it. Let's start with the very first thing: what does "time for a program" even look like as a picture?

---

Step 1 — What does "time to do a job" look like?

WHY draw it this way? Because a computer program that touches $N$ numbers has exactly this shape. If we can separate the "once" cost from the "per-number" cost, we can predict total time for any $N$ without running the code.

PICTURE. Below, the total time is a bar made of two pieces. The short dark stub on the left is the fixed setup. The long piece stretching right grows as we add more items — its length is (number of items) × (cost per item).

Putting the picture into symbols — read it left to right, exactly as the bar is drawn:

$T=\underset{\text{the dark stub, paid once}}{\underbrace{c_{\text{setup}}}}+\underset{\text{how many items}}{\underbrace{N}}\times \underset{\text{cost of one item}}{\underbrace{\tau }}$

Recall Why is this called a

linear model? Because if you double $N$, the long piece doubles but the stub stays fixed — the total time is a straight line when plotted against $N$ (a line = "linear"). ::: Straight-line growth: slope $\tau$, intercept $c_{\text{setup}}$.

---

Step 2 — The SAME job, two different workers

WHY? Because the whole story of vectorization is a story about $\tau$. A Python loop, for each number, must: unbox the object, check its type, pick the right method, and allocate a fresh result object (see parent note). A vectorized call does one raw CPU instruction per number. So we give them different names:

PICTURE. Two bars stacked. The Python bar's per-item pieces are fat; the vectorized bar's are thin. Notice the vectorized bar has a bigger setup stub — calling into C has its own fixed cost — but its per-item pieces are so thin the total is far shorter.

The two total times, term by term:

$T_{\text{py}}=c_{\text{setup}}^{\text{py}}+N{\tau }_{\text{py}},T_{\text{vec}}=\underset{\text{C-call overhead}}{\underbrace{c_{\text{setup}}}}+N\underset{\text{thin per item}}{\underbrace{{\tau }_{\text{vec}}}}$

---

Step 3 — Building the speedup ratio

WHY a ratio and not a difference? Because "faster" is naturally a multiplier. Saying "5 seconds faster" is meaningless without a baseline; "$100\times$ faster" is scale-free and tells you the whole story regardless of array size.

PICTURE. Take the two bars from Step 2 and lay the Python bar's length over the vectorized bar's length. How many vectorized-bars fit inside one Python-bar? That count is the speedup.

$\text{Speedup}=\frac{T_{\text{py}}}{T_{\text{vec}}}=\frac{\stackrel{\text{fat Python bar}}{\overbrace{N{\tau }_{\text{py}}}}}{\underset{\text{thin bar + its setup stub}}{\underbrace{c_{\text{setup}}+N{\tau }_{\text{vec}}}}}$

• Numerator $N{\tau }_{\text{py}}$: the whole Python cost — grows with $N$, slope ${\tau }_{\text{py}}$.
• Denominator $c_{\text{setup}}+N{\tau }_{\text{vec}}$: the vectorized cost — a fixed stub plus a thin slope.

This single fraction is the parent note's central result. The next steps ask: what does this fraction do as $N$ changes?

---

Step 4 — The tiny-$N$ case (where vectorization LOSES)

WHY this matters: People assume "vectorized is always faster." It is not. This step shows the degenerate case where the C-call setup is more expensive than just looping over 3 numbers in Python.

PICTURE. For $N=3$, the vectorized bar is almost all setup stub — barely shorter than (or even longer than) the tiny Python bar. The speedup fraction dips below 1, meaning "slower."

Set $N\to 0$ (the extreme degenerate case — an empty or near-empty array):

${\lim }_{N\to 0}\text{Speedup}=\frac{0\cdot {\tau }_{\text{py}}}{c_{\text{setup}}+0}=\frac{0}{c_{\text{setup}}}=0.$

• The numerator vanishes (no elements → no Python work saved).
• The denominator stays at $c_{\text{setup}}$ (you still pay to enter the C routine).
• Result: speedup $\to 0$, i.e. all cost, no benefit. This is why the parent warns: don't vectorize 4-element arrays.

---

Step 5 — The huge-$N$ case (where vectorization WINS)

WHY the limit? We want the best possible speedup — the ceiling. A "limit as $N\to \infty$" is the mathematical tool for "what value does this approach as we keep growing?" It answers: what's the most we can ever hope for? We use a limit (not just a big number) because it gives a clean, exact ceiling independent of any particular $N$.

PICTURE. As the bar grows rightward, the setup stub shrinks to a dot in comparison. The two per-item slopes (${\tau }_{\text{py}}$ vs ${\tau }_{\text{vec}}$) are all that's left — and their ratio is the whole answer.

Divide top and bottom of the fraction by $N$ (a legal move — it's multiplying by $\frac{1/N}{1/N}=1$), then send $N\to \infty$:

$\text{Speedup}=\frac{N{\tau }_{\text{py}}}{c_{\text{setup}}+N{\tau }_{\text{vec}}}=\frac{{\tau }_{\text{py}}}{\frac{c_{\text{setup}}}{N}+{\tau }_{\text{vec}}}\underset{N\to \infty }{\to }\frac{{\tau }_{\text{py}}}{0+{\tau }_{\text{vec}}}=\frac{{\tau }_{\text{py}}}{{\tau }_{\text{vec}}}.$

• The term $\frac{c_{\text{setup}}}{N}$: a fixed number divided by something huge → shrinks to 0. This is the key move — setup washes out.
• What survives: $\frac{{\tau }_{\text{py}}}{{\tau }_{\text{vec}}}$ — the ratio of the two per-item costs.

With ${\tau }_{\text{py}}\approx 100$ ns and ${\tau }_{\text{vec}}\approx 1$ ns:

$\frac{{\tau }_{\text{py}}}{{\tau }_{\text{vec}}}\approx \frac{100}{1}=100\times .$

That is where the famous "$100\times$" comes from — it is not magic, it is exactly the ratio of how expensive one Python element-step is versus one C element-step.

---

Step 6 — The whole curve, and the break-even point

WHY find break-even? Because it answers the practical question: how big must my array be before vectorizing pays off? We find it by setting speedup $=1$ (a tie) and solving for $N$.

Set $\frac{N{\tau }_{\text{py}}}{c_{\text{setup}}+N{\tau }_{\text{vec}}}=1$, cross-multiply, and collect $N$:

$N{\tau }_{\text{py}}=c_{\text{setup}}+N{\tau }_{\text{vec}}⟹N({\tau }_{\text{py}}-{\tau }_{\text{vec}})=c_{\text{setup}}⟹N_{\star }=\frac{c_{\text{setup}}}{{\tau }_{\text{py}}-{\tau }_{\text{vec}}}.$

• $N_{\star }$ = the break-even count. Below it, loop wins; above it, vectorize wins.
• Numerator $c_{\text{setup}}$: the bigger the fixed setup, the larger the array you need to justify it.
• Denominator ${\tau }_{\text{py}}-{\tau }_{\text{vec}}$: how much you save per element. Bigger savings → break even sooner.

Plug in $c_{\text{setup}}=1000$ ns, ${\tau }_{\text{py}}=100$, ${\tau }_{\text{vec}}=1$:

$N_{\star }=\frac{1000}{100-1}=\frac{1000}{99}\approx 10.1\text{ elements.}$

So past about 10 elements vectorization starts to win here.

PICTURE. The full S-shaped rise: below 1 on the left (red danger zone), crossing $y=1$ at $N_{\star }$, flattening at the dashed ceiling $100$.

---

The one-picture summary

Recall Feynman retelling — the whole walkthrough in plain words

Two workers do the same pile of sums. Python reads the full instruction sheet before touching each number — slow per number, but starts instantly. NumPy hires a fast C robot: the robot takes a moment to wake up (setup), then rips through the numbers almost for free. For a tiny pile the robot's wake-up time isn't worth it, so Python's head-start wins — that's why the speedup starts below 1. As the pile grows, the wake-up time becomes a rounding error, and all that's left is "how fast does each worker handle one number?" Python: ~100 ns. Robot: ~1 ns. Ratio: 100. So the speedup climbs and levels off at 100×. The exact size where the robot overtakes Python is the break-even $N_{\star }$ = wake-up cost ÷ savings-per-number ≈ 10 items here. No magic — just a fat cost versus a thin cost, watched as the pile grows.

---

Connections

• Vectorization — avoiding Python loops, speed comparison (Hinglish) — parent topic.
• NumPy Arrays vs Python Lists — why ${\tau }_{\text{vec}}$ is so small (packed typed memory).
• Universal Functions (ufuncs) — the compiled C kernels that give ${\tau }_{\text{vec}}$.
• Big-O Complexity — the linear $T=c+N\tau$ model is $O(N)$ per operation.
• Broadcasting Rules — vectorizing multi-dimensional combinations without loops.
• Numba and Cython — when even vectorization isn't enough, compile the loop directly.
• Memory Layout and Cache — why huge-$N$ speedups flatten (memory bandwidth).