Memory bandwidth and latency metrics
WHAT are we measuring?
HOW to derive bandwidth from first principles
We want: bytes moved per second. Build it from what physically happens each clock.
Step 1 — one transfer. Each transfer moves the bus width bytes. Why this step? The bus is wires; all bits flip together, so bytes cross per transfer.
Step 2 — transfers per second. The bus ticks at frequency (Hz = transfers/s for SDR). Why? One transfer per clock edge.
Step 3 — double data rate (DDR). DDR transfers on both rising and falling edges → factor of . Why? Two edges per clock cycle, so effective transfer rate .
Step 4 — parallel channels. independent channels each move bytes at once. Why? Channels are separate physical buses working simultaneously → they add.
Combining:
HOW to derive true (CAS) latency in nanoseconds
RAM latency is quoted as CL (CAS Latency) in clock cycles, but a cycle length shrinks as frequency rises. To compare fairly, convert to time.
Step 1 — cycle time. One clock cycle lasts seconds. Why? Frequency and period are reciprocals.
Step 2 — DDR naming trick. DDR "data rate" = . So the actual clock is half the advertised MT/s number.
Worked examples
Forecast-then-Verify prompt
Before reading answers: Predict — if you upgrade from 2 channels to 4 channels, what happens to (a) peak bandwidth and (b) CAS latency?
Recall Verify
(a) Bandwidth doubles (channels multiply). (b) Latency is unchanged — it's per-request physical access, independent of .
Flashcards
What is memory latency (with units)?
What is memory bandwidth (with units)?
Why are bandwidth and latency orthogonal?
Peak bandwidth formula?
Why the factor of 2 in DDR?
Convert CL cycles to nanoseconds?
Why can CL16@3200 and CL18@3600 have equal real latency?
DDR4-3200, 64-bit, dual-channel peak BW?
Little's Law for memory?
Does adding channels change latency?
Recall Feynman: explain to a 12-year-old
Imagine a water pipe from a tank to your cup. Latency = how long after you open the tap before the first drop reaches your cup — even a fat pipe has a delay if the tank is far away. Bandwidth = how many liters flow per second once water is moving — a fatter pipe pours more. Adding more pipes (channels) pours more water per second but the first drop still takes just as long. That's why fast RAM can feel "big" (bandwidth) yet still make you "wait" (latency).
Connections
- DRAM organization row buffer — where the physical access latency comes from.
- Cache hierarchy — caches exist to hide main-memory latency.
- DDR SDRAM generations — DDR3/4/5 double the transfer rate, not the latency.
- Littles Law — links throughput, latency, and concurrency.
- Memory channels and interleaving — how scales bandwidth.
- CPU memory wall — the growing gap: bandwidth improves faster than latency.
Concept Map
Hinglish (regional understanding)
Intuition Hinglish mein samjho
Dekho bhai, memory ke do alag-alag "speed" hote hain aur inhe kabhi mix mat karna. Latency matlab delay — request bhejne se lekar pehla byte aane tak kitna time laga (nanoseconds mein). Bandwidth matlab flow rate — ek second mein kitna data aa sakta hai (GB/s mein). Highway wala example socho: latency ek car ka pura safar time hai, bandwidth matlab ek second mein kitni cars nikal gayi. Dono alag cheezein hain — 20-lane wide highway (high bandwidth) par bhi commute lamba ho sakta hai (high latency).
Bandwidth ka formula first principles se banao: har transfer mein bus width bytes jaate hain, DDR mein dono clock edges par data jaata hai isliye , aur channels ek saath kaam karte hain, so . Isiliye DDR4-3200 ka "3200" already MT/s hai (yeh MHz hai), seedha use karo — factor of 2 dobara mat lagao, warna aadha answer aayega (yeh sabse common galti hai).
Latency ke liye CL (CAS Latency) cycles mein diya hota hai, but cycle ka time frequency par depend karta hai. Real time nikalne ke liye ns. Isi wajah se CL16@3200 aur CL18@3600 ka latency same 10 ns nikalta hai — high frequency har cycle ko chhota kar deti hai. Yaad rakho: DRAM ka physical access (row activate, column read) decades se zyada improve nahi hua, isiliye latency stuck hai but bandwidth badhta gaya — yahi "memory wall" problem hai. Little's Law () dono ko jodti hai: bus ko busy rakhne ke liye itna data hamesha pipeline mein hona chahiye.