Vector - SIMD instructions (SSE, AVX, NEON): Visual walkthrough

This page rebuilds the parent note's central result — how much faster does SIMD make your loop? — from absolute zero. We will earn every symbol before using it, and every step gets a blueprint figure you can point at.

If you have not yet seen the big idea, read the parent topic first. The one sentence you need to carry in: one instruction chews on several numbers at once. Everything below turns that sentence into a formula.

Definition Ground rules before we start

Two assumptions we will lean on the whole way, stated up-front so nothing is smuggled in:

The array is non-empty: $n \geq 1$ . If $n = 0$ there is nothing to process, both the scalar and SIMD times are $0$ , and the ratio $S = 0/0$ is undefined (and meaningless — you cannot speed up doing nothing). So we always take $n \geq 1$ , a whole number.
A vector instruction costs the same cycles as a scalar one. Call the scalar cost $t_{scalar}$ and the vector cost $t_{vector}$ . Real vector ALUs are built with $w$ parallel adders fed by one instruction, so a "multiply 8 floats" costs essentially the same as "multiply 1 float" — the work happens side by side, not one after another. We therefore set $t_{vector} = t_{scalar}$ and call the shared value simply $t$ . This single assumption is the engine of the whole speedup; if it failed, SIMD would buy you nothing.

Step 1 — Count the work the slow way (scalar)

WHAT. A scalar loop touches numbers one at a time. Imagine an array of $n$ numbers laid in a row. The machine walks the row, handling exactly one box per step.

WHY. Before we can say "SIMD is faster," we need a fair baseline: the honest cost of doing it one-at-a-time. You cannot measure "faster" without a "slower" to compare against.

PICTURE. Below, each amber box is one array element. The white arrow visits one box at a time. Count the arrow-visits — that is our workload.

Figure — Vector - SIMD instructions (SSE, AVX, NEON)

Let us name things carefully:

$n$ ::: how many numbers are in the array (here $n = 8$ ; always $n \geq 1$ ). It is just a count — no units.
$t$ ::: how many CPU cycles one element costs — this is the shared $t = t_{scalar} = t_{vector}$ from the ground rules. One box = $t$ ticks.

Multiply "number of boxes" by "ticks per box":

$T_{scalar} = how many elements n \cdot ticks each t$

$T_{scalar}$ is the total time (in cycles) for the scalar loop. That is our baseline.

Step 2 — Pack numbers into a wide register

WHAT. A vector register is a single storage slot wide enough to hold several numbers side by side. We call each side-by-side slot a lane. The number of lanes is the vector width $w$ .

WHY. This is the whole trick of SIMD. Instead of one number per hand, the CPU grabs $w$ numbers in one fist. We must define $w$ now because the speedup is going to be built entirely out of it.

PICTURE. One long cyan register split into $w = 4$ lanes, each holding one number. One load instruction fills all four lanes at once.

$w$ ::: vector width — how many numbers fit in one register at once. This depends on the hardware, not your data.

Concrete values from the parent note, so $w$ is not abstract:

Instruction set	Register width	Numbers ( $w$ ) for 32-bit floats
SSE (128-bit)	128 bits	$128/32 = 4$
AVX (256-bit)	256 bits	$256/32 = 8$
AVX-512 (512-bit)	512 bits	$512/32 = 16$

This ties straight to data parallelism: the same operation, many independent data items, no talking between lanes.

Step 3 — Count the work the fast way (SIMD)

WHAT. One SIMD instruction processes a whole register — all $w$ lanes — in the same $t$ cycles a scalar op took (that was our ground-rule assumption $t_{vector} = t_{scalar}$ ). So instead of $n$ single steps, we take groups of $w$ .

WHY. This is where the saving appears. If each instruction still costs $t$ cycles but now clears $w$ boxes instead of $1$ , we need far fewer instructions.

PICTURE. The same 8 boxes, but now bracketed into two groups of four. Each bracket is one instruction.

How many groups do we need? Divide the boxes by lanes: $n / w$ . In the picture $8/4 = 2$ groups. So a first guess is:

$T_{SIMD} \approx number of groups \frac{n}{w} \cdot ticks each group t$

But $n / w$ might not be a whole number — you cannot run "half an instruction." We fix that next.

Step 4 — The ceiling: partial groups still cost a full instruction

WHAT. The symbol $⌈ x ⌉$ (ceiling) means "round $x$ up to the next whole number." So $⌈ 2.0 ⌉ = 2$ , but $⌈ 1.25 ⌉ = 2$ and $⌈ 0.1 ⌉ = 1$ .

WHY. A register instruction fires on the whole register even if only one lane holds real data. Ten elements with $w = 8$ need $⌈ 10/8 ⌉ = ⌈ 1.25 ⌉ = 2$ instructions: one full group of 8, plus a second instruction for the leftover 2 (the other 6 lanes are wasted but still "along for the ride").

PICTURE. Ten boxes, $w = 4$ . Group 1 and Group 2 are full; Group 3 is only half-full but still costs one whole instruction (its empty lanes shaded amber = wasted).

So the honest SIMD time is:

$T_{SIMD} = groups, rounded up ⌈ \frac{n}{w} ⌉ \cdot ticks each t$

$⌈ n / w ⌉$ ::: the number of vector instructions actually issued, never a fraction.

Step 5 — Form the speedup ratio

WHAT. Speedup $S$ is defined as "old time divided by new time." If the new way takes half the time, $S = 2$ (twice as fast).

WHY. A ratio is the natural way to say "how many times faster." It cancels the unit "cycles," leaving a pure multiplier you can quote as " $8 \times$ ."

PICTURE. Two horizontal time-bars drawn to scale: the long scalar bar over the short SIMD bar. $S$ is literally "how many short bars fit inside the long one."

$S = \frac{T _{scalar}}{T _{SIMD}} = \frac{n \cdot t}{⌈ n / w ⌉ \cdot t}$

Notice the $t$ on top and bottom are the same $t$ — this is precisely our ground rule $t_{vector} = t_{scalar}$ : one element costs the same whether it rides alone or in a lane. So they cancel:

$S = \frac{n \cdot t}{⌈ n / w ⌉ \cdot t} = \frac{n}{⌈ n / w ⌉}$

The disappearance of $t$ is the key insight: speedup does not depend on how expensive the operation is — only on how many elements share a lane. (This cancellation is only legal because $t \neq = 0$ , which holds since $n \geq 1$ means real work exists.)

Step 6 — The large-array limit: why S → w

WHAT. When $n$ is much bigger than $w$ (written $n ≫ w$ ), the ceiling barely matters — one wasted partial group is a rounding blip on thousands of full groups. So $⌈ n / w ⌉ \approx n / w$ .

WHY. This gives the clean headline number people quote. We drop the ceiling because for big data it changes the answer by less than one instruction out of many.

PICTURE. A curve: $S$ plotted against $n$ for $w = 8$ . It sawtooths near the origin (ceiling effects) then flattens toward the dashed amber line $S = 8$ .

$S = \frac{n}{⌈ n / w ⌉} n ≫ w \frac{n}{n / w} = \frac{n \cdot w}{n} = w$

$S \approx w$

WHAT IT MEANS. The best you can hope for is a speedup equal to the number of lanes: $4 \times$ on SSE, $8 \times$ on AVX, $16 \times$ on AVX-512. This is the ideal that Loop Vectorization and Auto-vectorization chase.

Step 7 — Edge and degenerate cases

WHAT. The formula $S = n / ⌈ n / w ⌉$ is exact and covers every input with $n \geq 1$ . Let us walk the corners so nothing surprises you.

WHY. The parent note quotes " $4 \times$ – $16 \times$ " as the ideal but " $3 \times$ – $6 \times$ " in reality. The gap lives entirely in these edge cases. A reader who only saw $S \approx w$ would be shocked by real benchmarks; we defuse that now.

PICTURE. Four mini-panels, one per case, each showing the box layout and its $S$ .

Case $n = 0$ (empty array). Ruled out by our ground rule — no work, no speedup, $S = 0/0$ is undefined. Nothing to draw.
Case $n = w$ (exactly one full register). $S = w / ⌈ 1 ⌉ = w /1 = w$ . Full speed, no waste.
Case $n < w$ (fewer elements than lanes, e.g. $n = 3, w = 8$ ). $S = 3/ ⌈ 3/8 ⌉ = 3/1 = 3$ . You still issue one instruction but only $3$ lanes carry real data — speedup is $3$ , not $8$ . Tiny arrays cannot reach the ideal.
Case $n = 1$ (a single element). $S = 1/ ⌈ 1/ w ⌉ = 1/1 = 1$ . No speedup at all — SIMD degenerates to scalar, and the setup overhead can make it slower.
Case "not divisible" (e.g. $n = 10, w = 8$ ). $S = 10/2 = 5$ , below the ideal $8$ because of the wasted lanes in the second instruction (the remainder problem from the parent note).

The reasons real speedup lands at $3$ – $6$ instead of $w$ — memory bandwidth, unaligned loads, scalar tails — are the practical ceiling on top of this arithmetic ceiling. Memory bandwidth in particular ties into Cache Optimization: if the data cannot arrive fast enough, extra lanes sit idle.

The one-picture summary

WHAT. One blueprint compressing the entire chain: the scalar row of $n$ boxes, the grouping into $⌈ n / w ⌉$ registers, the two time-bars, and the ratio that survives after $t$ cancels — ending at $S \approx w$ .

WHY. So the whole argument lives in your head as a single image, not seven scattered ones. When you later recall "why is SIMD $w$ times faster?", this picture should replay: same work, fewer presses, time-per-press cancels, only lane-count remains. That is the payload of the entire page in one glance.

Recall Feynman retelling

Picture a row of chocolates you have to wrap. Wrapping them one by one takes as many moves as there are chocolates — that's $T_{scalar} = n \cdot t$ , where $t$ is the time for one wrap. (And you need at least one chocolate, $n \geq 1$ — wrapping an empty row is not "faster," it is just nothing.)

Now you get a magic tray that wraps $w$ chocolates in a single press — and crucially, one press takes the same time as wrapping one chocolate by hand, because the tray works all its slots at once. You load $w$ at a time, so you only press the button $⌈ n / w ⌉$ times — rounding up because even a half-full tray needs a whole press. That's $T_{SIMD}$ .

"How many times faster?" is just the old time over the new time. The time-per-press $t$ appears in both, so it cancels — meaning it does not matter whether wrapping is quick or slow, only how many fit on the tray. You are left with $S = n / ⌈ n / w ⌉$ .

If you have loads of chocolates, the odd leftover tray barely matters and the answer settles at exactly the tray size, $S \approx w$ : an 8-lane tray is 8× faster. But with just one chocolate, the tray gives you nothing ( $S = 1$ ), and with a leftover handful the wasted tray slots drag you below the ideal. That gap — plus real-world snags like data arriving too slowly — is why the shiny "8×" becomes a solid "5×" in practice.

Recall Quick self-check

Why does $t$ cancel in the speedup? ::: Because we assumed a vector instruction costs the same $t$ cycles as a scalar one ( $t_{vector} = t_{scalar}$ ); the ratio keeps only the count of elements vs. instructions, so $t$ divides out. For $n = 10$ , $w = 8$ , what is $S$ ? ::: $S = 10/ ⌈ 10/8 ⌉ = 10/2 = 5$ . Why is the ideal speedup exactly $w$ ? ::: Because with $n ≫ w$ , $⌈ n / w ⌉ \approx n / w$ , and $n \div (n / w) = w$ . What happens when $n = 1$ ? ::: $S = 1/ ⌈ 1/ w ⌉ = 1$ — no speedup; SIMD collapses to scalar. Why must $n \geq 1$ ? ::: For $n = 0$ both times are $0$ and $S = 0/0$ is undefined — you cannot speed up doing no work.