Foundations — Simultaneous multithreading (SMT - hyperthreading)

This page builds — from absolute zero — every word and symbol the Simultaneous multithreading (SMT - hyperthreading) parent note leans on. Read it top to bottom: each idea is the floor the next one stands on. Nothing is used before it is drawn.

1. What is a "clock cycle"?

Everything a processor does is chopped into equal ticks of a metronome. One tick = one clock cycle. Inside one tick the chip can start (and often finish) a small amount of work.

Look at the figure: time runs left to right, split into equal boxes. Every box is one cycle. When we later say "IPC" or "500 cycles", we are simply counting boxes. That is the only clock we will ever mean.

2. What is an "instruction" and an "execution unit"?

An instruction is one tiny order to the CPU: add these two numbers, load this value from memory, jump here if zero.

An execution unit is the physical circuit that carries out one kind of order. Real cores have several:

• ALU (Arithmetic Logic Unit) — does integer add / subtract / compare.
• FPU (Floating-Point Unit) — does decimal-number maths.
• Load/Store unit — moves data between the core and memory.

The figure shows 4 tools (boxes) over several cycles. A green box = a tool doing real work this cycle. A grey box = a tool sitting idle. The whole point of SMT is: stop making grey boxes.

3. The symbol $N$ — how many execution units

We write $N$ for the number of execution units (tools) on the bench. The parent note uses $N=4$.

4. The symbol IPC — Instructions Per Cycle

Now the single most important symbol on the whole topic.

If a program runs 1000 instructions in 500 cycles:

Why IPC and not "speed in GHz"? Clock speed (GHz) tells you how fast the metronome ticks. IPC tells you how much you get done per tick. SMT does not speed up the metronome — it raises IPC by filling grey boxes. So IPC is exactly the right lens for this topic.

5. Utilization — the fraction of tools kept busy

With IPC $=1.5$ and $N=4$:

6. Why boxes go grey: stalls, dependencies, misses

Three reasons a tool sits idle. Each is a prerequisite topic in its own right.

(a) Data dependency. Instruction $I_2$ needs the answer of $I_1$. Until $I_1$ finishes, $I_2$ cannot start — the tool waits. (Formally studied in Pipeline Hazards.)

(b) Cache miss. The data a load needs is not in the fast on-chip memory, so the core must fetch it from far-away DRAM — 50 to 200 cycles of waiting. Understanding where data lives is the Memory Hierarchy; keeping shared copies consistent is Cache Coherence. A stall is any such forced wait.

(c) Branch misprediction. The core guesses which way an if will go to keep working (Branch Prediction). A wrong guess throws away the work started down the wrong path — more grey boxes.

The figure shows one thread's timeline: a burst of green, then a long grey stall (cache miss), then green again. That grey gap is pure wasted hardware — and it is exactly where a second thread's instructions could go.

7. Thread, thread context, and "replicated vs shared"

A thread is one independent stream of instructions — one program's worth of orders in order.

To run two threads on one core, the core must remember two of certain small things:

These are cheap to duplicate (~5% extra silicon). Everything expensive — the execution units ($N$ tools), caches, branch predictor — stays shared. That asymmetry (duplicate the tiny bookkeeping, share the big machinery) is the whole trick that makes SMT affordable.

Running multiple independent threads to get more work done is called Thread-Level Parallelism (TLP); SMT is simply TLP implemented inside one core, per cycle.

8. Throughput vs latency — the two clocks you must not confuse

9. The symbol $f$ and the speedup model

The parent's performance formula uses one final symbol:

Reading it in plain words: during the busy fraction $(1-f)$ the tools are already full, so a second thread adds nothing — that term stays as-is. During the stalled fraction $f$, the second thread fills those gaps, halving the wasted time — hence $f/2$. The denominator is "how much time the work now really takes"; one over it is the speedup.

With $f=0.6$:

Notice the two extremes: $f=0$ (never stalls) gives speedup $=1$ (no gain), and $f=1$ (always stalls) gives $1/0.5=2\times$ (the theoretical max). Every real workload sits between.

Prerequisite map

Equipment checklist

Test yourself — you are ready for the parent note when you can answer each without peeking.