SIMD — vector instructions, SSE - AVX — Hinglish notes

4.1.24 · Coding › Computer Architecture (Deep)

SIMD exist kyun karta hai?

YEH PROBLEM SOLVE KARTA HAI: har element par instruction overhead aur under-used wide datapaths.

HARDWARE KAISE KARTA HAI: register file wider ho jaati hai (128/256/512 bits). addps jaisi instruction register ko independent lanes mein cut karti hai aur saare adders parallel mein fire hote hain — koi carry lane boundaries cross nahi karta.

Definitions

Figure — SIMD — vector instructions, SSE - AVX

Ek vector add actually kaise execute hota hai

Worked example 8 floats ke liye AVX ke saath

c[i] = a[i] + b[i]

// intrinsics (compiler-portable SIMD)
__m256 va = _mm256_loadu_ps(&a[i]);   // load 8 floats (unaligned ok)
__m256 vb = _mm256_loadu_ps(&b[i]);   // load 8 floats
__m256 vc = _mm256_add_ps(va, vb);    // 8 adds in ONE instruction
_mm256_storeu_ps(&c[i], vc);          // store 8 floats

loadu kyun, load kyun nahi? _mm256_load_ps ko 32-byte-aligned address chahiye warna fault hota hai; loadu koi bhi address tolerate karta hai (historically thoda slower, modern CPUs par equal). 8 ek saath kyun process karte hain? $256/32 = 8$ lanes — yahi natural stride hai. Remainder loop kyun chahiye: agar array length 8 ka multiple nahi hai, toh last len % 8 elements ek scalar tail loop se kiye jaate hain.

Worked example Poora vectorized loop (tail ke saath)

size_t i = 0;
for (; i + 8 <= n; i += 8) {                 // Why: only step while a full vector fits
    __m256 va = _mm256_loadu_ps(a + i);
    __m256 vb = _mm256_loadu_ps(b + i);
    _mm256_storeu_ps(c + i, _mm256_add_ps(va, vb));
}
for (; i < n; ++i) c[i] = a[i] + b[i];       // Why: scalar tail for leftover elements

Worked example Horizontal reduction — array ko sum karna

Lane-wise ops aasaan hain; lanes ke across combine karna ("horizontal") tricky part hai.

__m256 acc = _mm256_setzero_ps();
for (size_t i = 0; i + 8 <= n; i += 8)
    acc = _mm256_add_ps(acc, _mm256_loadu_ps(a + i)); // Why: keep 8 partial sums
// Ab 8 partial sums -> 1 fold karo
__m128 lo = _mm256_castps256_ps128(acc);      // lanes 0..3
__m128 hi = _mm256_extractf128_ps(acc, 1);    // lanes 4..7
__m128 s  = _mm_add_ps(lo, hi);               // 4 partial sums
s = _mm_hadd_ps(s, s);                         // pairwise add
s = _mm_hadd_ps(s, s);                         // phir -> total in lane 0
float total = _mm_cvtss_f32(s);

Loop mein har lane sum karne ki jagah fold kyun? 8 independent accumulators rakhne se ek long dependency chain avoid hoti hai → CPU additions pipeline karta hai; expensive cross-lane fold sirf ek baar end mein pay karte hain.

Steel-manned mistakes

Active recall

Recall Lanes ki number kaise compute karte hain? (cover karo aur answer do)

$lanes = \frac{register width in bits}{element size in bits}$ . E.g. AVX (256) with f32 (32) → 8 lanes.

Recall Real speedup lane count ke barabar kyun nahi hota?

Amdahl's law (non-vectorizable fraction) + memory bandwidth ceiling + setup/tail overhead + possible downclocking.

Recall

addps aur addss mein kya difference hai? addps = packed, saari lanes add karta hai; addss = scalar, sirf lane 0.

Recall SIMD sum mein multiple accumulators kyun rakhte hain?

Dependency chain todne ke liye taaki additions pipeline hon; cross-lane fold sirf ek baar end mein karo.

Recall Feynman: SIMD ko ek 12-saal ke bachche ko explain karo

Socho ek teacher papers grade kar rahi hai. Normally woh ek ek paper grade karti hai. SIMD ek magic stamp ki tarah hai jo 8 stamp wide hai — woh 8 papers line mein lagaati hai aur ek press mein sab par "+5" stamp kar deti hai. Action (stamp +5) same hai; sirf papers alag hain. Toh woh 8× faster finish karti hai — jab tak woh pile se 8 papers fast enough pakad sake (wahi memory limit hai!).

Forecast-then-verify drill

Flashcards

SIMD ka full form kya hai?

Single Instruction, Multiple Data — ek instruction kai packed data elements par operate karti hai ek saath.

SSE / AVX / AVX-512 ke register widths?

128-bit / 256-bit / 512-bit.

Har ek ke liye register file names?

xmm (SSE, 128), ymm (AVX, 256), zmm (AVX-512, 512).

Lanes ki number ka formula?

lanes = register_width_bits / element_size_bits.

AVX ke saath f32 mein kitni lanes?

256/32 = 8.

AVX-512 ke saath int8 mein kitni lanes?

512/8 = 64.

addps aur addss mein difference?

packed (saari lanes) vs scalar (sirf lane 0).

Real speedup lane count se kam kyun hota hai?

Amdahl's law, memory bandwidth limit, setup/tail overhead, possible downclocking.

Amdahl speedup formula?

1 / ((1-f) + f/lanes), f = vectorizable fraction.

Scalar tail loop kyun chahiye?

array length lane count ka multiple nahi ho sakti; leftover elements ek ek karke done hote hain.

'Horizontal' operation kya hota hai?

ek operation jo lanes ke across values combine kare (jaise saari lanes sum karna), lane-wise vertical ops ke opposite.

Reduction mein multiple SIMD accumulators kyun rakhte hain?

dependency chain todne ke liye taaki additions pipeline hon; fold sirf ek baar end mein karo.

vzeroupper kis kaam aata hai?

costly SSE/AVX (non-VEX/VEX) state-transition penalty se bachne ke liye.

SIMD float reductions scalar se alag kyun ho sakte hain?

float addition associative nahi hai; lane-grouping adds ka order change karti hai → alag rounding.

loadu vs load (aligned)?

load ko alignment chahiye (warna fault); loadu koi bhi address allow karta hai.

Connections

Data-Level Parallelism — SIMD in-core form of DLP hai.
Amdahl's Law — achievable speedup cap karta hai.
Memory Bandwidth and the Roofline Model — compute-bound vs memory-bound decide karta hai ki SIMD kitna payoff dega.
Cache and Alignment — aligned loads, cache lines vs vector width.
Auto-vectorization and the Compiler — -O3 -march=native, -ffast-math.
GPU SIMT — GPU cousin (Single Instruction, Multiple Threads).
Floating Point Representation — FP addition ki non-associativity.