Interconnects, Buses & SoC
Level: 3 (Production — from-scratch derivations, code-from-memory, explain-out-loud) Time limit: 45 minutes Total marks: 60
Question 1 — PCIe Bandwidth Derivation (12 marks)
PCIe uses different encoding schemes across generations. Gen1/2 use 8b/10b encoding; Gen3/4/5 use 128b/130b encoding.
(a) Derive from first principles the usable unidirectional bandwidth (in GB/s) of a PCIe 4.0 x16 link. State the raw transfer rate per lane, the encoding overhead factor, and show all arithmetic. (6 marks)
(b) A GPU accelerator needs 50 GB/s of unidirectional host-to-device bandwidth. Determine the minimum PCIe generation and lane width combination (from x1, x4, x8, x16) that satisfies this. Show your reasoning for at least two candidate configurations. (4 marks)
(c) Explain why the industry moved from 8b/10b to 128b/130b encoding. What problem does line encoding solve in a SerDes link? (2 marks)
Question 2 — SerDes and Signaling (10 marks)
(a) Explain out loud (in writing) why modern high-speed interconnects use serial signaling rather than wide parallel buses. Give at least three distinct physical/electrical reasons. (6 marks)
(b) PCIe 6.0 switched from NRZ to PAM4 signaling. Explain what PAM4 is, and how it doubles data rate without doubling the symbol (baud) rate. State one drawback. (4 marks)
Question 3 — AXI Handshake (code-from-memory) (12 marks)
(a) Describe the AXI VALID/READY handshake rule. State precisely when a transfer occurs and the ordering constraint on when VALID may be asserted. (4 marks)
(b) Write, from memory, a small SystemVerilog (or clear pseudo-RTL) snippet for a simple AXI-Stream source that drives TVALID, TDATA, and observes TREADY, sending a byte counter value each accepted beat. Include the handshake logic. (6 marks)
(c) Name the five independent channels of full AXI4 and state their direction (master→slave or slave→master). (2 marks)
Question 4 — DMA & CPU Offload Derivation (10 marks)
A CPU must move a 4 MB buffer from a peripheral to memory.
- Programmed I/O (PIO): CPU copies 4-byte words, 5 CPU cycles per word.
- DMA: CPU spends 200 cycles to program the transfer; DMA runs autonomously.
- CPU clock = 2 GHz.
(a) Compute the CPU cycles consumed under PIO. (3 marks) (b) Compute the CPU cycles consumed to set up DMA, and the speedup in CPU-occupancy vs PIO. (4 marks) (c) Explain what cycle-stealing vs burst mode DMA means, and one implication of DMA for cache coherency in an SoC. (3 marks)
Question 5 — NoC Topology & Scaling (10 marks)
(a) For a 2D mesh NoC connecting nodes, derive the network diameter (max hop count between two nodes) as a function of . (4 marks)
(b) Compare a ring, a 2D mesh, and a crossbar for a 64-node interconnect in terms of diameter and approximate link/switch cost. Fill in and justify. (4 marks)
(c) Why do AMD's Infinity Fabric and Intel's mesh use tiled/mesh topologies rather than a single monolithic crossbar at high core counts? (2 marks)
Question 6 — CXL & Coherent Interconnect (explain-out-loud) (6 marks)
(a) CXL defines three sub-protocols: CXL.io, CXL.cache, CXL.mem. Briefly state the purpose of each. (3 marks)
(b) Explain how CXL leverages the PCIe physical layer yet provides something PCIe alone cannot for memory expansion / accelerators. (3 marks)
Answer keyMark scheme & solutions
Question 1 — PCIe Bandwidth
(a) (6 marks)
- PCIe 4.0 raw rate per lane = 16 GT/s (Giga-transfers/s). (1)
- Encoding = 128b/130b → efficiency factor = . (1)
- Usable per lane = Gbit/s. (1)
- Convert to bytes: GB/s per lane. (1)
- x16 → GB/s per direction. (2)
(b) (4 marks) Per-lane usable bandwidth (GB/s):
| Gen | GT/s | enc | GB/s/lane |
|---|---|---|---|
| 3.0 | 8 | 128/130 | 0.985 |
| 4.0 | 16 | 128/130 | 1.969 |
| 5.0 | 32 | 128/130 | 3.938 |
- Need 50 GB/s.
- Gen4 x16 = 31.5 GB/s → insufficient. (1)
- Gen5 x16 = GB/s → sufficient. (1)
- Gen5 x8 = → insufficient. (1)
- Minimum satisfying: PCIe 5.0 x16 (63 GB/s). (Gen4 x16 fails; no x32 standard.) (1)
(c) (2 marks)
- 8b/10b has 25% overhead; 128b/130b has ~1.5% → far more efficient at high rates. (1)
- Line encoding ensures DC balance and sufficient transitions so the receiver's clock-recovery (CDR) stays locked (no long runs of 0s/1s) and enables running-disparity/error detection. (1)
Question 2 — SerDes
(a) (6 marks, 2 each for any three)
- Skew: parallel lanes must arrive bit-aligned; at high rates trace-length differences cause skew that's hard to control. Serial removes inter-lane skew per bit.
- Crosstalk / EMI: many parallel switching lines interfere; fewer differential pairs reduce this.
- Pin/trace count & cost: serial needs far fewer pins/wires, cheaper connectors and PCB routing.
- Clock/data skew: parallel needs a separate synchronous clock; serial embeds clock via CDR, avoiding clock-to-data skew.
- (Also: signal integrity at high frequency favors point-to-point differential serial.)
(b) (4 marks)
- PAM4 = Pulse Amplitude Modulation, 4 levels → each symbol carries 2 bits (00,01,10,11) via 4 voltage levels. (2)
- At the same baud rate it transmits 2 bits/symbol instead of 1 (NRZ), so PCIe 6.0 hits 64 GT/s at ~32 Gbaud. (1)
- Drawback: smaller voltage margin between levels → worse SNR, needs FEC and equalization. (1)
Question 3 — AXI
(a) (4 marks)
- Transfer occurs on a rising clock edge when both VALID and READY are HIGH. (2)
- Master may assert VALID independent of READY, but once VALID is asserted it must remain asserted until the transfer completes (VALID cannot be withdrawn before READY). READY may be asserted before or after VALID. (2)
(b) (6 marks) — example accepted if handshake correct:
module axis_source (
input logic clk, rst_n,
output logic tvalid,
output logic [7:0] tdata,
input logic tready
);
logic [7:0] cnt;
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
cnt <= 8'd0;
tvalid <= 1'b1; // always have data
end else begin
// transfer happens when tvalid && tready
if (tvalid && tready)
cnt <= cnt + 8'd1; // advance only on accepted beat
end
end
assign tdata = cnt;
endmoduleMarks: valid/ready handshake condition (2), advance on accept only (2), correct reset & data drive (2).
(c) (2 marks) — 0.4 each:
- AW (write address) M→S
- W (write data) M→S
- B (write response) S→M
- AR (read address) M→S
- R (read data + response) S→M
Question 4 — DMA
Buffer = 4 MB = bytes → words = words.
(a) (3 marks)
- PIO cycles = cycles. (3)
(b) (4 marks)
- DMA setup CPU cycles = 200 cycles. (1)
- Speedup (CPU occupancy) = . (2)
- (Wall-clock time for data still limited by DMA/memory bandwidth, but CPU is free during transfer.) (1)
(c) (3 marks)
- Cycle-stealing: DMA grabs the bus one (or few) transfers at a time, interleaving with CPU accesses; burst mode: DMA holds the bus for a whole block, faster but stalls CPU. (2)
- Cache coherency implication: DMA writes to memory may leave stale cache lines; SoC must snoop/invalidate or use coherent interconnect (e.g., AXI ACE/CHI) — else software must flush/invalidate. (1)
Question 5 — NoC
(a) (4 marks)
- In a mesh, max Manhattan distance is hops in X plus in Y. (2)
- Diameter . (2)
(b) (4 marks) — 64 nodes:
| Topology | Diameter | Cost |
|---|---|---|
| Ring | 64 links, cheap | |
| 2D mesh (8×8) | ~2N links, moderate | |
| Crossbar | 1 | crosspoints, expensive |
Justification marks: ring diameter N/2 (1), mesh (1), crossbar diameter 1 (1), cost ordering (1).
(c) (2 marks)
- Crossbar area/power grows as and becomes unroutable/timing-infeasible at high core counts; mesh/tiled fabric scales ~linearly, offers modular tiling and localized short wires with acceptable diameter. (2)
Question 6 — CXL
(a) (3 marks, 1 each)
- CXL.io: PCIe-equivalent I/O — discovery, config, DMA, interrupts (mandatory base).
- CXL.cache: lets a device coherently cache host memory.
- CXL.mem: lets the host access device-attached memory as coherent, load/store addressable memory (memory expansion).
(b) (3 marks)
- CXL runs on the same PCIe electrical/PHY (Flex Bus), reusing SerDes and slots. (1)
- But it adds cache coherency and load/store memory semantics with low latency — PCIe alone is non-coherent, DMA/message-based. (1)
- Enables memory pooling/expansion and coherent accelerators sharing host address space. (1)
[
{"claim":"PCIe4 x16 usable ~31.5 GB/s","code":"bw=16*(128/130)/8*16; result=abs(bw-31.5)<0.2"},
{"claim":"PCIe5 x16 usable ~63 GB/s meets 50GB/s","code":"bw=32*(128/130)/8*16; result=bw>50 and abs(bw-63.0)<0.3"},
{"claim":"PCIe5 x8 fails 50GB/s","code":"bw=32*(128/130)/8*8; result=bw<50"},
{"claim":"PIO cycles=5242880","code":"words=4*2**20//4; result=(words*5)==5242880"},
{"claim":"DMA speedup=26214.4","code":"words=4*2**20//4; result=abs(words*5/200-26214.4)<0.1"},
{"claim":"8x8 mesh diameter=14","code":"k=8; result=2*(k-1)==14"},
{"claim":"64-node ring diameter=32","code":"result=64//2==32"}
]