Memory bandwidth and latency metrics: Exercises — Hardware deep dive

Before we start, here is the toolkit in one place, in words:

Keep this picture in mind for every problem — it separates the "how much per second" road from the "how soon" road.

Figure — Memory bandwidth and latency metrics

Level 1 — Recognition

Exercise 1.1

A datasheet lists a memory kit as "DDR4-3200 CL16". Which of these two numbers is a time-to-first-byte quantity and which is a throughput quantity? Give the units of each.

Recall Solution

3200 is the data rate in MT/s (megatransfers per second). It feeds the bandwidth (throughput) formula — units of transfers/s, and after multiplying by width and channels, bytes/s.
CL16 is the CAS Latency: a count of clock cycles. On its own it is dimensionless, but converted to real time it gives a latency in nanoseconds.
So: 3200 → bandwidth road (how much per second); CL16 → latency road (how soon). They are orthogonal.

Exercise 1.2

True or false: "A 4-channel kit always has lower CAS latency than a 2-channel kit." Explain in one line.

Recall Solution

False. Channels ( $N$ ) only multiply bandwidth; the number $N$ does not appear anywhere in the latency formula $t_{CL} = CL / f_{clk}$ . Latency is a per-request physical property (row activate → column read), untouched by how many parallel buses you own.

Level 2 — Application

Exercise 2.1

DDR5-6000, single channel, bus width 64 bits. Compute the peak bandwidth in GB/s.

Recall Solution

Width in bytes: $64 bits \div 8 = 8$ bytes. Why? 8 bits make 1 byte, so 64 data wires carry 8 bytes per transfer.
Data rate: $6000$ MT/s $= 6.0 \times 1 0^{9}$ transfers/s (the "6000" already includes the ×2 of DDR).
Channels: $N = 1$ . $BW = 8 \times 6.0 \times 1 0^{9} \times 1 = 4.8 \times 1 0^{10} bytes/s = 48.0 GB/s .$

Exercise 2.2

A kit is CL30 at 6000 MT/s. Find the real CAS latency in nanoseconds.

Recall Solution

Use $t_{CL} = \frac{2 CL}{data rate (MT/s)} \times 1 0^{3}$ ns. $t_{CL} = \frac{2 \times 30}{6000} \times 1 0^{3} = \frac{60}{6000} \times 1 0^{3} = 0.01 \times 1 0^{3} = 10.0 ns .$ Sanity check the "÷2 clock": real clock $f_{clk} = 6000/2 = 3000$ MHz, so one cycle is $1/3000 MHz = 0.333$ ns; $30 \times 0.333 = 10.0$ ns. ✓

Exercise 2.3

DDR4-2666, dual channel, 64-bit bus. Peak bandwidth?

Recall Solution

$W = 8$ bytes, data rate $= 2.666 \times 1 0^{9}$ transfers/s, $N = 2$ . $BW = 8 \times 2.666 \times 1 0^{9} \times 2 = 4.2656 \times 1 0^{10} \approx 42.66 GB/s .$

Level 3 — Analysis

Exercise 3.1

Kit A: CL16 @ 3200 MT/s. Kit B: CL18 @ 3600 MT/s. Which responds sooner (lower real latency)?

Recall Solution

Kit A: $t = \frac{2 \times 16}{3200} \times 1 0^{3} = \frac{32}{3200} \times 1 0^{3} = 10.0$ ns.
Kit B: $t = \frac{2 \times 18}{3600} \times 1 0^{3} = \frac{36}{3600} \times 1 0^{3} = 10.0$ ns. Tie — identical real latency (10.0 ns). The extra cycle of CL18 is exactly cancelled by the shorter cycle time at 3600. But Kit B has higher bandwidth (bigger data rate), so B is the better all-round kit despite the "bigger CL".

Exercise 3.2

Kit C: CL14 @ 3200 MT/s. Kit D: CL16 @ 3200 MT/s. Same data rate. Which is faster to respond, and by how many ns?

Recall Solution

Same data rate → same cycle time $= \frac{2}{3200} \times 1 0^{3} = 0.625$ ns per cycle.

Kit C: $14 \times 0.625 = 8.75$ ns.
Kit D: $16 \times 0.625 = 10.0$ ns. Kit C is faster by $10.0 - 8.75 = 1.25$ ns. Here CL does decide the winner, because frequency is held constant so cycles no longer shrink to compensate.

Exercise 3.3

You double the data rate but the datasheet also lists double the CL cycles. What happens to real latency? Show it algebraically.

Recall Solution

Let old kit be $(CL, R)$ with real latency $t_{0} = \frac{2 CL}{R} \times 1 0^{3}$ . New kit is $(2 CL, 2 R)$ : $t_{1} = \frac{2 ( 2 CL )}{2 R} \times 1 0^{3} = \frac{2 CL}{R} \times 1 0^{3} = t_{0} .$ Latency is unchanged — the doubled cycle count is exactly divided out by the doubled rate. This is the algebra behind why "higher MT/s but higher CL" so often lands on the same nanoseconds.

Level 4 — Synthesis

Exercise 4.1 (Little's Law bridge)

A system sustains 51.2 GB/s bandwidth and has main-memory latency 60 ns. How many bytes must be "in flight" (outstanding) at once to keep the bus fully busy?

Recall Solution

Little's Law for memory: outstanding bytes $= B \times L$ (see Littles Law). Why this product? If the bus delivers $B$ bytes every second and each request's data takes $L$ seconds to start arriving, you must have $B \times L$ bytes already requested and traveling, or the bus will idle waiting for the next request. $B \times L = 51.2 \times 1 0^{9} \frac{bytes}{s} \times 60 \times 1 0^{- 9} s = 3072 bytes .$ If a cache line is 64 bytes, that is $3072/64 = 48$ outstanding requests needed to saturate the bus.

Exercise 4.2

Same 60 ns latency, but you upgrade to 4 channels of DDR4-3200 (64-bit). (a) New bandwidth? (b) New required outstanding bytes to saturate? (c) Did latency change?

Recall Solution

(a) One channel: $8 \times 3.2 \times 1 0^{9} = 25.6$ GB/s. Four channels: $25.6 \times 4 = 102.4$ GB/s.
(b) Outstanding $= 102.4 \times 1 0^{9} \times 60 \times 1 0^{- 9} = 6144$ bytes ( $= 96$ cache lines of 64 B).
(c) Latency is unchanged at 60 ns — $N$ appears in the bandwidth formula only. Doubling the road's width does not shorten one car's commute. Lesson: more bandwidth raises the concurrency you must supply to actually use it. Bandwidth is a potential; Little's Law tells you the price in outstanding requests.

Exercise 4.3

A CPU core can keep at most 10 requests in flight (64-byte lines each) to a memory system with 80 ns latency. What effective bandwidth can this single core actually reach, regardless of the bus's peak?

Recall Solution

Rearrange Little's Law: $B_{effective} = \frac{outstanding bytes}{L}$ .

Outstanding bytes $= 10 \times 64 = 640$ bytes.
$B_{effective} = \frac{640 bytes}{80 \times 1 0 ^{- 9} s} = 8.0 \times 1 0^{9} bytes/s = 8.0 GB/s .$ Even if the bus peaks at 51.2 GB/s, this one core is concurrency-limited to 8 GB/s. To use the rest of the bus you need more cores/requests — a real bottleneck behind the CPU memory wall.

Level 5 — Mastery

Exercise 5.1 (design & defend)

You must pick ONE kit for a latency-sensitive workload (many small, dependent random reads — pointer chasing, where each read must finish before the next can be issued):

Kit X: DDR5-6400 CL40

Kit Y: DDR4-3600 CL16

(a) Compute both real latencies. (b) Compute both single-channel 64-bit peak bandwidths. (c) Which do you choose, and why does the workload decide it?

Recall Solution

(a) Latencies:

Kit X: $t = \frac{2 \times 40}{6400} \times 1 0^{3} = \frac{80}{6400} \times 1 0^{3} = 12.5$ ns.
Kit Y: $t = \frac{2 \times 16}{3600} \times 1 0^{3} = \frac{32}{3600} \times 1 0^{3} \approx 8.89$ ns.

(b) Bandwidths (single channel, $W = 8$ B):

Kit X: $8 \times 6.4 \times 1 0^{9} = 51.2$ GB/s.
Kit Y: $8 \times 3.6 \times 1 0^{9} = 28.8$ GB/s.

(c) Choose Kit Y. In pointer chasing each access depends on the previous one, so only one request is in flight at a time. By Little's Law the achievable bandwidth is tiny either way ( $\approx 64 B / t$ ), so the extra peak bandwidth of Kit X is wasted. What matters is time-to-first-byte, and Kit Y is faster (8.89 ns vs 12.5 ns). The workload's low concurrency is what flips the decision away from the "bigger number" kit.

Exercise 5.2 (mastery synthesis)

For Kit Y above (8.89 ns latency), pointer-chasing with exactly 1 outstanding 64-byte request: what effective bandwidth do you actually reach, and what fraction of its 28.8 GB/s peak is that?

Recall Solution

Effective $B = \frac{64 bytes}{8.89 \times 1 0 ^{- 9} s} \approx 7.2 \times 1 0^{9} = 7.2 GB/s$ .
Fraction of peak $= 7.2/28.8 = 0.25 = 25%$ . Even the "fast-latency" kit uses only a quarter of its bus when concurrency is 1. This is the whole reason we quote two metrics and why caches (see Cache hierarchy) and Memory channels and interleaving exist — to raise concurrency and hide latency so the bandwidth is not stranded.

Recall One-line self-test

Why do we quote latency and bandwidth instead of a single "speed"? ::: Because they answer different questions (how soon vs how much per second) and Little's Law shows your real throughput is capped by whichever your workload's concurrency runs into first.

Connections

Memory bandwidth and latency metrics — the parent topic these exercises drill.
Littles Law — the bridge used in every L4/L5 problem: outstanding = BW × latency.
DDR SDRAM generations — where the CL and MT/s numbers in these kits come from.
Memory channels and interleaving — how $N$ scales the bandwidth answers.
Cache hierarchy — the practical way real systems raise concurrency and hide latency.
CPU memory wall — why concurrency-limited single-core bandwidth (Ex 4.3, 5.2) is a growing problem.
DRAM organization row buffer — the physical source of the CAS latency we keep converting.