FPGA-based acceleration: Visual walkthrough — Hardware deep dive

This is the beating heart of FPGA-based acceleration: an FPGA is fast not because its clock is fast, but because it can lay out a spatial pipeline and keep it full. Let us earn that result from zero.

Step 1 — What is a "stage"? (the assembly line picture)

WHAT. Imagine building a toy car. The work splits into 3 jobs: (1) attach wheels, (2) paint, (3) box it. Each job sits at its own station. A worker at a station does only that one job, forever.

WHY. A CPU is one worker doing all three jobs on one car before touching the next — one person, three tools, lots of walking. A pipeline is three workers, each glued to one station, passing the car down the line. This "one station per job" idea is exactly what an FPGA does in silicon: one physical circuit per operation. We split work into stages so each station can work on a different car at the same moment.

PICTURE. Below, the three stations sit in a row. A car enters the left, exits the right. Right now only station 1 is busy — the pipe is still empty behind it.

Step 2 — What is a "clock tick"? (the drumbeat)

WHAT. Every station passes its car to the next station at the same instant — on a shared drumbeat. That beat is the clock. The time between two beats is the clock period, written $T$ .

WHY. All stations must hand off together, or a half-finished car would collide with the next one. So the drum can only beat as fast as the slowest station can finish its job. If painting takes longest at $t_{ma x}$ seconds, then

$T = t_{ma x}$

$T$ — seconds between beats (the clock period). Smaller $T$ = faster clock.
$t_{ma x}$ — the time the slowest stage needs. It sets the pace; a chain is only as fast as its slowest link.

PICTURE. Three stations with little stopwatches. Painting (station 2) is the slow one, so its stopwatch defines the drumbeat $T$ for everyone.

Step 3 — The naive way: no pipeline (one car at a time)

WHAT. First, count time the slow way — no assembly line. One car goes through all $S$ stations, start to finish, before the next car even enters.

WHY. We need a baseline to compare against. This is how a CPU would do it: finish item 1 completely, then item 2, and so on. We call the number of items $M$ .

The count, term by term. One item takes $S$ stages, each costing one beat $T$ , so one item = $S T$ . Do that for all $M$ items:

$Time_{naive} = M \cdot S \cdot T$

$M$ — how many data items (cars) we push through.
$S$ — stages per item (each item still needs all of them).
$T$ — seconds per stage.
The product $M \cdot S$ = total stage-visits; times $T$ = total seconds.

PICTURE. A tall stack of cars waiting. Car 2 cannot start until car 1 has left all three stations. Two of the three stations sit idle at any moment — wasted workers.

Step 4 — The pipeline trick: overlap the cars

WHAT. Now let every station stay busy. The moment car 1 leaves station 1, car 2 enters station 1 — while car 1 is at station 2. The cars overlap in time.

WHY. Those idle workers from Step 3 were the waste. Overlapping means all $S$ stations are working on $S$ different cars once the line is full. This is the whole idea: fill the pipe, then never let a station idle.

PICTURE — the space-time chart. This is the key diagram. Rows = stations, columns = clock ticks. Each colored block is one car occupying one station for one tick. Read the diagonal stripes: each car marches down-right, one station per tick. After the pipe fills, every column is fully colored — every station busy every tick.

Step 5 — Count the pipelined time (fill, then one-per-tick)

WHAT. Count the ticks in the diagram of Step 4. Two phases: filling the pipe, then draining one result each tick.

WHY. The first result is not free — car 1 must still crawl through all $S$ stations before any car finishes. That is the fill cost. But every car after the first pops out just one tick later, because they are all overlapped.

The count, built up.

Filling the pipe (first result exits): it takes $S$ ticks for car 1 to pass all $S$ stations.

$first result out S$

The remaining $M - 1$ cars each finish exactly one tick after the car before it (they are overlapped by exactly one beat):

$fill S + one per remaining item (M - 1) = S + M - 1 ticks$

Each tick is $T$ seconds, so:

$Time_{pipe} = (S + M - 1) T$

$S$ — the one-time fill cost (ticks before the first result appears).
$M - 1$ — the extra ticks, one for each item after the first.
$T$ — seconds per tick, multiplying the whole tick count into seconds.

PICTURE. The same chart, now with the two phases shaded: a pale-yellow "fill" triangle of length $S$ on the left, then a chalk-blue "steady state" where results drip out one per column.

Step 6 — The speedup, and why it heads toward $S$

WHAT. Divide the naive time (Step 3) by the pipelined time (Step 5).

WHY. Speedup is "how many times faster" — old time over new time. This single ratio tells us whether the whole exercise was worth it.

The algebra, term by term.

$Speedup = \frac{Time _{naive}}{Time _{pipe}} = \frac{M S T}{( S + M - 1 ) T}$

The $T$ 's cancel (same clock in both) — pipelining is not about a faster clock:

$Speedup = \frac{M S}{S + M - 1}$

Numerator $M S$ — the total work (item-stage visits).
Denominator $S + M - 1$ — the actual ticks we paid.
Their ratio = useful-work-per-tick vs one-tick-of-clock.

Now push $M$ (the batch size) toward infinity. Divide top and bottom by $M$ :

$\frac{M S}{S + M - 1} = \frac{S}{\frac{S - 1}{M} + 1} M \to \infty \frac{S}{0 + 1} = S$

$\frac{S - 1}{M}$ — the fill cost spread over $M$ items. As $M$ grows, this shrinks to $0$ .
What remains is just $S$ : a full $S \times$ speedup.

PICTURE. A curve of speedup vs $M$ : it starts near $1$ (tiny batch, all fill), climbs, and flattens as it hugs the dashed ceiling line at $S$ .

Step 7 — Edge cases: the corners where intuition slips

WHAT. We must check the small and degenerate inputs, or the reader will trust the formula where it lies.

WHY. A formula you cannot break is a formula you cannot trust. Four corners:

(a) One item, $M = 1$ . Plug in: $Speedup = \frac{1 \cdot S}{S + 1 - 1} = \frac{S}{S} = 1$ . No speedup at all! With a single car, the pipeline is pure fill — you pay the full $S$ ticks and gain nothing. This is latency vs throughput: latency for one item is unchanged.

(b) One stage, $S = 1$ . $Speedup = \frac{M \cdot 1}{1 + M - 1} = \frac{M}{M} = 1$ . Nothing to overlap — one station is already the whole job. Pipelining needs $S \geq 2$ to help.

(c) Zero items, $M = 0$ . Time $= (S + 0 - 1) T = (S - 1) T$ — the pipe drains the leftovers already inside, but the naive time is $0$ . Speedup is undefined ( $0/0$ ); physically, "no work, no win." Guard against dividing by an empty batch.

(d) Unbalanced stages. If one stage is far slower, $T = t_{ma x}$ is dragged up by that stage and every other station idles part of each tick. The formula still holds, but the clock is bad. The real-world fix is balancing — splitting the slow stage so no station dominates $T$ .

PICTURE. Four mini-panels, one per corner: single car (all fill), single station (no overlap), empty batch, and a lopsided line where the fat slow station stalls the drum.

Step 8 — Put real numbers in (the parent's example, checked)

WHAT. Use $S = 12$ stages, $M = 10, 000$ items, clock $200$ MHz so $T = 5$ ns.

WHY. A formula must survive contact with numbers.

$cycles = S + M - 1 = 12 + 9999 = 10011$ $Time_{pipe} = 10011 \times 5 ns \approx 50.06 μ s$

Naive baseline at $1$ ns per op, $12$ ops per item:

$Time_{naive} = 10000 \times 12 \times 1 ns = 120 μ s$

$Speedup = \frac{120000}{10011} \approx 11.99 \approx 12 = S ✓$

We landed a hair below the ceiling $S = 12$ — exactly the fill toll $\frac{S - 1}{M} = \frac{11}{10000}$ we predicted in Step 6. The picture and the algebra agree.

Recall Say the whole limit in one line

What is $lim_{M \to \infty} \frac{M S}{S + M - 1}$ and why? ::: It is $S$ ; the one-time fill cost $S - 1$ spreads over $M$ items and vanishes, leaving the full stage count as the achievable speedup. With $M = 1$ , what is the speedup? ::: Exactly $1$ — a single item is pure fill, no overlap to exploit.

The one-picture summary

This final chalkboard compresses the whole story: the naive stack (workers idle) versus the overlapped pipeline (all workers busy), the fill triangle of cost $S - 1$ , the steady state dripping one result per tick, and the speedup curve climbing to its ceiling at $S$ .

Recall Feynman retelling — explain it to a friend with no math

Imagine building toy cars. The slow way: one worker builds a whole car — wheels, paint, box — then starts the next. Three cars = nine slow jobs done one after another.

The pipeline way: three workers, each glued to one station. The instant car 1 leaves the wheel station, car 2 rolls in. Soon all three workers are busy on three different cars at once. After that, a finished car pops out on every drumbeat.

There's a catch at the start: the very first car still has to crawl through all three stations before anything comes out — that's the "fill." You pay that toll once. But over a thousand cars, paying it once is nothing. So a three-station line, over a long run, cranks out cars almost three times faster than the lone worker — and a twelve-station line, almost twelve times. The magic number is just how many stations you have, $S$ — not how fast the drum beats. That is why a slow-clocked FPGA with a deep pipeline flattens a fast CPU: it finishes one whole computation's worth of work on every single tick.

Related build-up: Pipelining and Throughput · Configurable Logic Blocks (LUTs & Flip-Flops) · Dataflow architectures · Roofline Model & Arithmetic Intensity · GPU vs FPGA vs ASIC trade-offs