Memory bandwidth and latency metrics: Visual walkthrough

We assume you know nothing yet. Let's earn each piece.

Step 1 — One wire carries one bit

WHAT. The smallest unit of the whole story is a single wire — a metal line between the memory chip and the CPU. At any instant it holds one bit: a voltage that is either "high" (call it $1$ ) or "low" (call it $0$ ).

WHY start here. Everything else — bytes, buses, bandwidth — is just many wires and many instants stacked up. If we understand one wire in one instant, we can multiply our way to the full formula and never lose track of a factor.

PICTURE. Below, the blue line is the wire. The yellow dot is the voltage level right now. Flip it high or low and you have sent one bit.

Figure — Memory bandwidth and latency metrics

Step 2 — Eight wires side by side make one byte; $W$ bytes make the bus

WHAT. Put $8$ wires next to each other and let them all flip at the same instant. Together they carry $8$ bits $=$ one byte. A real memory bus has many such wires; we bundle them into $W$ bytes of width.

WHY. The CPU rarely wants a lone bit — it wants a chunk. Widening the road means more data crosses per instant without needing more time. This is the first place a number multiplies our bandwidth, so we name it $W$ .

PICTURE. Each blue line is a wire; a group of $8$ is bracketed as one byte. The whole stack is the bus width $W$ (here $W = 8$ bytes $= 64$ wires — a typical single channel).

bytes per transfer W [bytes]

Here $W$ is doing one job: telling us how fat each single transfer is. It has no time in it yet — that comes next.

Step 3 — Time enters: the clock ticks $f$ times a second

WHAT. The bus doesn't send data whenever it likes. A clock — a voltage that rises and falls like a heartbeat — paces it. The clock repeats $f$ times per second. $f$ is the frequency, measured in hertz ( $Hz =$ ticks per second).

WHY this tool, and why now. Bandwidth is a rate — data per second. A rate needs a clock to count "per second" against. Width alone ( $W$ ) told us how much per transfer; frequency tells us how often a transfer happens. Multiplying them is what turns a static width into a flowing rate.

PICTURE. The yellow wave is the clock. One full up-and-down is one cycle; its length in time is the period $T_{clk}$ . The green dots mark the rising edges — one data transfer fires on each in plain (single-data-rate) memory.

transfers per second (SDR) = one transfer per rising edge f

So far, single-data-rate bandwidth would be $W \times f$ : fat road $\times$ how often it fires.

Step 4 — Double data rate: use both edges of the clock

WHAT. Modern DDR ("Double Data Rate") memory sends data on the rising edge AND the falling edge of each clock cycle. Two transfers per cycle instead of one.

WHY the factor of 2. A clock cycle has exactly two edges. Wasting the falling edge is wasting half the clock. By firing on both, the effective transfer rate becomes $2 f$ without raising the clock frequency (which is hard and hot to do). That is precisely where the mysterious $\times 2$ in the parent formula is born.

PICTURE. Same yellow clock, but now green dots sit on both the rising and falling edges. Count them in one cycle: two. Two data words leave per cycle.

data rate = both edges 2 \times clock Hz f [transfers/s]

Step 5 — Parallel channels: several buses at once

WHAT. A memory channel is a whole independent bus (its own $W$ wires, its own clock behaviour). Motherboards run $N$ of them side by side, all moving data at the same time.

WHY they add (multiply). Channels are physically separate roads. They do not share wires, so their throughputs simply stack: $N$ channels move $N$ times the data every instant. This is the third and last multiplier.

PICTURE. Two blue buses (channel 0, channel 1), each with its own clock and its own byte-stack, both flowing simultaneously into the CPU. Nothing waits on anything.

Now assemble the three pieces we earned — width, doubled rate, channels:

Step 6 — Latency is a different road: rebuild it from the clock's period

WHAT. Latency is time to the first byte, not a rate. RAM quotes it as CL (CAS Latency) — a count of clock cycles you must wait after asking for a column of data. To compare kits fairly we turn cycles into nanoseconds.

WHY a new derivation. Bandwidth counted "how many transfers per second". Latency asks "how long is the wait itself". Same clock, opposite question — so we use the period $T_{clk} = 1/ f$ (seconds per cycle), not the rate. Multiply the wait-count by the length of one wait.

PICTURE. A timeline of clock cycles. The red bracket spans CL cycles — the stall before the first data word (green) arrives. Each cycle is $T_{clk}$ seconds long.

Step 7 — Edge and degenerate cases (never get surprised)

WHAT / WHY / PICTURE. Let's push every knob to its limit so no scenario ambushes you.

$N = 1$ (single channel). Bandwidth halves versus dual; the $N$ factor is just $1$ . Latency is unchanged — a single request still waits CL cycles.
$N \to$ more channels. Bandwidth grows linearly. Latency stays flat — it is a per-request physical delay (see DRAM organization row buffer), independent of how many roads exist.
$f \to$ very high. Bandwidth rises and each cycle shortens. But CL usually rises too (the physical array didn't get faster), so $t_{CL}$ barely moves — this is the CPU memory wall.
CL $= 0$ (degenerate). $t_{CL} = 0$ : impossible physically, but the formula correctly says "no wait". It confirms latency is purely the cycle-count times cycle-length.
Idle bus (Little's Law limit). If too few requests are in flight, the road sits empty and you never reach $BW_{peak}$ .

The last point is the bridge between the two roads. To keep a bandwidth $B$ busy across a latency $L$ , you must have this many bytes already in flight:

outstanding bytes = rate B \times time L .

PICTURE. Left: bandwidth grows step-by-step as $N$ increases, while the latency line stays perfectly flat. Right: the Little's Law rectangle — area $B \times L$ — is the amount of data that must be en route.

The one-picture summary

Everything above compressed: width $\times$ doubled rate $\times$ channels builds the bandwidth road (green), while CL $\times$ period builds the separate latency road (red). They share only the clock — and stay orthogonal.

Recall Feynman retelling — the whole walkthrough in plain words

Start with one wire carrying one bit. Lay $8$ side by side and they carry a byte in one shot; a whole bundle of them is the width $W$ — how fat the road is. Now add a clock ticking $f$ times a second so we can say "per second". DDR memory is clever: it fires on both the up and the down of every tick, so the real transfer rate is $2 f$ — that's the famous $\times 2$ . Bolt several independent roads together and you get $N$ channels, multiplying the flow. Stack those three multipliers and you have $BW = W \times 2 f \times N$ bytes per second.

Latency is a different question on the same clock. Instead of "how many per second", it asks "how long is the wait". Take one cycle's length ( $1/ f$ seconds), multiply by the number of cycles you must stall (CL), and you get the real wait in nanoseconds. That's why a slower-labelled kit with fewer wait-cycles can respond just as fast, and why adding channels helps bandwidth but never shortens the wait. Keep enough data in flight (Little's Law: $B \times L$ bytes) or the fat road sits empty.

Connections

DRAM organization row buffer — the physical source of the CL wait in Step 6.
Cache hierarchy — exists to hide the latency road of Step 6–7.
DDR SDRAM generations — each generation grows $2 f$ (Step 4), not CL.
Littles Law — the $B \times L$ rectangle of Step 7.
Memory channels and interleaving — the $N$ multiplier of Step 5.
CPU memory wall — why bandwidth (Steps 2–5) outruns latency (Step 6).