Noesis
4.1.24 · D5Computer Architecture (Deep)

Question bank — SIMD — vector instructions, SSE - AVX

1,830 words8 min read↑ Back to topic

Prerequisite lenses you'll lean on: Data-Level Parallelism, Amdahl's Law, Memory Bandwidth and the Roofline Model, Cache and Alignment, Auto-vectorization and the Compiler, Floating Point Representation, and the contrast with GPU SIMT.

Ground vocabulary first (so the questions land)

Before the traps, pin down the words every item below leans on. Read the picture, then the definitions.

Figure — SIMD — vector instructions, SSE - AVX
Figure — SIMD — vector instructions, SSE - AVX

True or false — justify

True or false: doubling the vector width from 128 to 256 bits always halves the runtime of an array loop.
False. It only halves the arithmetic portion; if the loop is bounded by memory bandwidth the bytes must still all be fetched, so wider registers move data no faster.
True or false: addps and addss differ only in speed.
False. addps (packed) writes all lanes; addss (scalar) writes only lane 0 and leaves the upper lanes untouched — a correctness difference, not just speed.
True or false: SIMD gives free speedup because ALU lanes are cheap to replicate.
Partly false. Replicating lanes is cheap in silicon, but the usable speedup is throttled by Amdahl's Law, bandwidth, alignment, and tail-loop overhead — the "free" throughput is a ceiling you rarely touch.
True or false: a carry can ripple from lane 0 into lane 1 during a packed integer add.
False. Lanes are independent by design — no carry crosses a lane boundary. That independence is why the adds can fire in parallel in one instruction.
True or false: summing an array with SIMD and with scalar code gives bit-identical results for floats.
False. Float addition is not associative; lane-grouped partial sums reorder the additions and round differently. Integers, being exact, do match.
True or false: AVX-512 always beats AVX2 because it has twice the lanes.
False. Heavy AVX-512 use can trigger frequency downclocking; a "wider" loop running at a lower clock can net-lose on short or mixed workloads. Benchmark, don't assume.
True or false: because you write intrinsics instead of assembly, SIMD code is fully portable across compilers and vendors.
False. Intrinsics like _mm256_add_ps are x86-specific and their exact availability/behaviour depends on the compiler and target flags; ARM uses NEON/SVE intrinsics instead. They're more portable than raw asm, but not universally standardized.
True or false: horizontal (cross-lane) operations are just as cheap as lane-wise ones.
False. Lane-wise ops run fully in parallel; cross-lane ops like hadd or shuffles move data between lanes and are slower — the whole reduction idiom exists to do them only once, at the end.
True or false: the dedicated hadd instruction is always the fastest way to fold lanes.
False. On several microarchitectures hadd decodes to multiple µops; a hand-written permute + add sequence can be lower-latency. The right choice is microarch-dependent — measure.
True or false: -ffast-math is required for the compiler to auto-vectorize a float sum reduction.
True. Because reassociation changes rounding, a standards-conforming compiler won't reorder float additions unless you opt out of strict semantics. See Auto-vectorization and the Compiler.

Spot the error

A loop uses _mm256_load_ps(a + i) on an array from malloc. What's the trap?
_mm256_load_ps (aligned) faults unless the address is 32-byte-aligned; plain malloc gives only 16-byte alignment. Use loadu, or allocate with alignment. See Cache and Alignment.
Someone writes for (i = 0; i <= n; i += 8) to vectorize 8-wide. Bug?
The guard must be i + 8 <= n, not i <= n — otherwise the last iteration reads/writes past the end of the array when n isn't a clean multiple of 8.
A vectorized loop has no scalar tail loop after it. When does it break?
Whenever n is not a multiple of the lane count: the final n % lanes elements are never processed. The tail loop exists precisely to mop up that remainder.
Code does acc = _mm256_add_ps(acc, v) but forgets _mm256_setzero_ps() before the loop. Consequence?
acc starts as garbage stack bits, so the sum is off by an arbitrary amount. Accumulators must be explicitly zeroed.
A vectorized loop over int32 sums into a signed accumulator and the values overflow. Correctness trap?
Signed integer overflow is undefined behaviour in C/C++ — the compiler may assume it never happens and mis-optimize. Use unsigned types (well-defined wraparound) or a wider accumulator.
Hand-written asm mixes VEX-encoded AVX with legacy (non-VEX) SSE and omits vzeroupper. Cost?
An expensive state-save/restore penalty on every transition between the two encodings, silently tanking throughput. Compilers insert vzeroupper; hand code must too.
A reduction stores into acc inside the loop and reads it next iteration with one accumulator. Performance bug?
It creates a serial dependency chain — each add waits for the previous result. Using several independent accumulators lets the pipeline overlap adds; fold them once at the end.
float total = _mm256_castps256_ps128(acc); — why won't this compile as a sum?
cast just reinterprets the low 128 bits as an __m128; it does no addition. You still must add the high half, then horizontally fold, then extract a scalar with _mm_cvtss_f32.
A loop uses a gather (_mm256_i32gather_ps) to load 8 scattered indices as if it were free vector speedup. Trap?
Gather/scatter are not free — each element is a separate memory access, so latency and throughput are far worse than a contiguous loadu. They pay off only when the alternative is fully scalar, not versus a linear load.

Why questions

Why does SIMD amortize instruction cost but scalar code doesn't?
One decoded instruction drives many lanes, so fetch/decode is paid once per vector instead of once per element — that is Data-Level Parallelism in action.
Why can a perfectly-vectorized loop still be far from lanes×, even in theory?
Amdahl's Law: only the vectorizable fraction speeds up, so — the serial remainder caps you.
Why is a memory-bound loop barely helped by SIMD?
The bottleneck is bytes-per-second from RAM, not adds-per-second. Faster ALUs just wait longer on loads — see the Roofline.
Why keep 8 partial sums during the loop and fold only afterward?
To break the dependency chain so additions pipeline; the expensive cross-lane fold is paid once, not every iteration.
Why does SIMD differ from GPU SIMT even though both run one op on many data?
SIMD is explicit lanes in one thread (you write the vector width); SIMT is many scalar threads the hardware groups into warps, with per-lane divergence handling. Same DLP idea, different programming model.
Why does the lane count come out to width / element_size?
Packing lays elements end-to-end with no gaps, so you simply count how many fixed-size elements fit in a fixed-width register.

Edge cases

Edge case: array length n = 0. What runs?
Neither the vector loop (i + 8 <= 0 is false) nor the tail (i < 0 false) executes — result stays the initial value (e.g. 0 for a sum). Correct, no special-casing needed.
Edge case: n = 5 on an 8-wide loop. What happens?
The vector body runs zero times (0 + 8 <= 5 false); all 5 elements go through the scalar tail. SIMD helps only past the first full vector.
Edge case: element size equals register width (e.g. one 512-bit value in a 512-bit register).
lanes = 512/512 = 1 — SIMD degenerates to a single scalar-like op with no parallelism benefit.
Edge case: array crosses a cache-line/page boundary mid-vector with an unaligned load.
loadu still works correctly but may split into two accesses, costing extra cycles — an alignment/cache concern, not a correctness bug.
Edge case: a float reduction over an array containing NaN.
The NaN propagates through every partial-sum lane it touches; the final total is NaN regardless of lane grouping — order doesn't rescue it. See Floating Point Representation.
Edge case: a loop produces denormal (subnormal) float results near zero.
Denormals can trigger a huge per-operation slowdown (microcode assist) on many CPUs. Enabling flush-to-zero / denormals-are-zero (FTZ/DAZ) restores speed at the cost of tiny-value precision. See Floating Point Representation.
Edge case: summing a huge array in f32 by keeping several lane accumulators and folding at the end.
This approximates pairwise (tree) summation, whose error grows like rather than the naive sum's — usually more accurate, but only if the accumulators are combined as a balanced tree; a lopsided fold gives no such guarantee.