Data hazards and forwarding - bypassing — Hardware notes

What Are Data Hazards?

Why RAW Hazards Dominate

In a classic 5-stage pipeline (IF → ID → EX → MEM → WB):

Instruction 1: ADD R1, R2, R3    # R1 = R2 + R3
Instruction 2: SUB R4, R1, R5    # R4 = R1 - R5  (needs R1 from I1)

Timeline without forwarding:

Clock: 1    2    3    4    5    6    7
ADD:   IF   ID   EX   MEMWB
SUB:        IF   ID   EX   MEM  WB
              ↑
         Reads R1 here (cycle 3)
         But R1 written in cycle 5!

The SUB instruction reads R1 in the ID stage (cycle 3), but ADD writes R1 in the WB stage (cycle 5). Two-cycle gap = wrong data.

Formula Data Hazard Detection Condition

A RAW hazard exists between instructions $i$ and $j$ (where $j$ follows $i$ ) if:

$\text{Destination}(i) = \text{Source}(j) \land \text{Destination}(i) \neq \text{R0}$

Why each part?

$\text{Destination}(i) = \text{Source}(j)$ : Instruction $j$ reads what $i$ writes
$\neq \text{R0}$ : Register R0 is hardwired to zero in many architectures (MIPS, RISC-V), so writes to it don't create real hazards Expanded for pipeline stages: Hazard exists if instruction in ID stage reads a register that:
Instruction in EX stage will write (1 cycle away)
Instruction in MEM stage will write (2 cycles away)
Instruction in WB stage is currently writing (handled by design)

Solution1: Stalling (The Naive Fix)

Worked example Stalling Example

Problem Code:

ADD R1, R2, R3    # I1: R1 = R2 + R3
SUB R4, R1, R5    # I2: R4 = R1 - R5  (hazard!)

Stall Solution: Insert pipeline bubles (NOPs) until data is ready.

Clock: 1    2    3    4    5    6    7    8
ADD:   IF   ID   EX   MEM  WB
SUB:        IF   [ID] [ID] [ID] ID   EX   MEM
                 stall stall stall

Why this works: By cycle 6, ADD has written R1 to the register file (cycle 5 WB completes), so SUB can safely read it in cycle 6.

Cost: 3 wasted cycles = 3 bubles = 3× slowdown for this sequence.

Derivation of stall cycles needed:

Data produced in EX stage (for ALU ops) → available in cycle $n + 2$ (EX at $n$ , MEM at $n+1$ , WB at $n+2$ )
Data needed in ID stage by dependent instruction
If dependent instruction is in cycle $m$ , it needs data by cycle $m + 1$ (ID stage)
Stalls required: $(n + 2) - (m + 1) = n - m + 1$

For consecutive instructions ( $m = n + 1$ ): Stalls = $n - (n+1) + 1 = 0$ stalls? Wrong! This ignores that data isn't written to register file until WB completes.

Correct calculation: If instruction $i$ is in EX at cycle $n$ , result written at cycle $n + 2$ (end of WB). If instruction $i+1$ is in ID at cycle $n+1$ , it reads at cycle $n+1$ . Gap = $(n+2) - (n+1) = 1$ cycle... but register file read happens before clock edge where write occurs. Actual gap = 2 cycles for ALU-to-ALU dependency.

Solution 2: Forwarding/Bypassing (The Smart Fix)

Forwarding Paths: The Architecture

Key components:

Forwarding unit (hazard detection + control logic)
Multiplexers at ALU inputs (select between register file data vs. forwarded data)
Pipeline registers that carry result values forward (EX/MEM, MEM/WB)

Derivation of forwarding logic:

Start with the hazard condition: instruction in EX stage needs data from instruction in EX, MEM, or WB stage.

Case 1: EX-to-EX forwarding (1 cycle away)

ADD R1, R2, R3    # Cycle n: EX stage computes R1
SUB R4, R1, R5    # Cycle n: ID stage, cycle n+1: EX stage needs R1

At cycle $n+1$ :

ADD result is in EX/MEM pipeline register (just came out of EX stage)
SUB is in EX stage, needs R1 for ALU input

Forward from EX/MEM.ALUOutput → ALU input MUX.

Case 2: MEM-to-EX forwarding (2 cycles away)

ADD R1, R2, R3    # Cycle n: EX, cycle n+1: MEM
AND R6, R7, R8    # Cycle n+1: EX (doesn't use R1)
SUB R4, R1, R5    # Cycle n+2: EX stage needs R1

At cycle $n+2$ :

ADD result is in MEM/WB pipeline register
SUB is in EX stage

Forward from MEM/WB.ALUOutput → ALU input MUX.

Priority rule derivation: If both EX/MEM and MEM/WB have data for the same register, which do we forward?

ADD R1, R2, R3    # I1: Cycle n+1: MEM stage (result in MEM/WB)
ADD R1, R6, R7    # I2: Cycle n+1: EX stage (result in EX/MEM)
SUB R4, R1, R5    # I3: Cycle n+1: ID stage, cycle n+2: EX stage

Both I1 and I2 write R1, but I2 is more recent. I3 should use I2's result.

Rule: EX/MEM forwarding (more recent) overides MEM/WB forwarding.

Worked example Forwarding Example: ALU-to-ALU

ADD R1, R2, R3    # I1
SUB R4, R1, R5    # I2: needs R1 from I1
AND R6, R1, R7    # I3: needs R1 from I1
OR  R8, R1, R9    # I4: needs R1 from I1

Timeline with forwarding:

Clock:  1    2    3    4    5    6    7
ADD:    IF   ID   EX   MEM  WB
SUB:         IF   ID   EX   MEM  WB
                       ↑
                  Forward from EX/MEM
AND:              IF   ID   EX   MEM  WB
                            ↑
                       Forward from MEM/WB
OR:                    IF   ID   EX   MEM  WB
                               ↑
                       Read from register file (no hazard)

Why each step?

SUB (cycle 4): ADD is in MEM stage, result in EX/MEM register → forward to SUB's EX stage ALU input
AND (cycle 5): ADD is in WB stage, result in MEM/WB register → forward to AND's EX stage ALU input
OR (cycle 6): ADD has completed WB, R1 is in register file → normal read, no forwarding needed

Performance: Zero stalls! Original3-cycle penalty eliminated.

Load-Use Hazards: When Forwarding Isn't Enough

Worked example Load-Use Hazard

LW  R1, 0(R2)     # I1: Load R1 from memory
ADD R4, R1, R3    # I2: needs R1 (load-use hazard!)

Timeline:

Clock:  1    2    3    4    5    6
LW:     IF   ID   EX   MEM  WB
ADD:         IF   ID   [ID] EX   MEM
                       stall

Why the stall?

LW produces data at end of MEM stage (cycle 4)
ADD needs data at start of EX stage (cycle 4 without stall)
Even with forwarding, data isn't ready until cycle 4, but ADD needs it in cycle 3

Solution: Insert 1 stall cycle, then forward from MEM/WB register.

Why this step? The memory read happens during the MEM stage, so data isn't available until MEM completes. No forwarding path can retrieve data before it exists.

Formula for load-use stalls:

1 & \text{if next instruction uses load result} \\ 0 & \text{if one instruction between load and use} \end{cases}$$

Worked example Compiler Optimization: Instruction Scheduling

Original code (has load-use hazard):

LW  R1, 0(R2)
ADD R4, R1, R3    # Stall here!
SUB R5, R6, R7

Optimized code (reordered by compiler):

LW  R1, 0(R2)
SUB R5, R6, R7    # Independent instruction fills the gap
ADD R4, R1, R3    # No stall! R1 is ready

Why this works: By the time ADD reaches EX stage (cycle 5), LW has completed MEM stage (cycle 4), and data can be forwarded from MEM/WB register. The SUB instruction "fills the load delay slot."

Performance gain: 1 fewer stall per load-use pair in critical loops = significant speedup.

Forwarding Unit Implementation

Derivation of control logic:

For ForwardA (ALU input 1, corresponds to Rs):

if (EX/MEM.RegWrite 
    and EX/MEM.RegisterRd ≠ 0 
    and EX/MEM.RegisterRd = ID/EX.RegisterRs)
then ForwardA = 10

else if (MEM/WB.RegWrite 
         and MEM/WB.RegisterRd ≠ 0
         and MEM/WB.RegisterRd = ID/EX.RegisterRs
         and not(EX/MEM hazard condition))  // Priority check
then ForwardA = 01

else ForwardA = 00

Why the priority check? If both EX/MEM and MEM/WB match, we want the more recent value (EX/MEM), so MEM/WB forwarding only happens when EX/MEM doesn't match.

ForwardB logic is identical, but compares with ID/EX.RegisterRt.

Worked example Double Hazard

ADD R1, R2, R3    # I1
ADD R1, R1, R4    # I2: overwrites R1
ADD R5, R1, R6    # I3: which R1? (should be I2's result)

Timeline:

Clock:  1    2    3    4    5
I1:     IF   ID   EX   MEM  WB
I2:          IF   ID   EX   MEM
I3:               IF   ID   EX

At cycle 5:

I1's result is in MEM/WB.RegisterRd (= R1)
I2's result is in EX/MEM.RegisterRd (= R1)
I3 is in EX stage, needs R1

Which to forward? I2's result (more recent).

Why this works: The priority rule ensures EX/MEM forwarding overides MEM/WB forwarding. This handles the "most recent write" semantics correctly.

Performance Impact Analysis

Formula CPI with Hazards

Starting from ideal CPI = 1 (one instruction per cycle), we add penalties: $\text{CPI}_{\text{actual}} = 1 + \text{stalls per instruction}$

For a program with:

$f_{\text{load}}$ = fraction of instructions that are loads
$p_{\text{use}}$ = probability that next instruction uses load result
$f_{\text{branch}}$ = fraction of branches (control hazards, separate topic) $\text{CPI}_{\text{stall-only}} = 1 + f_{\text{load}} \cdot p_{\text{use}} \cdot 1$

With forwarding: Eliminate ALU-to-ALU stalls, only load-use stalls remain.

Typical values: $f_{\text{load}} \approx 0.25$ , $p_{\text{use}} \approx 0.5$ (without compiler scheduling)

$\text{CPI} = 1 + 0.25 \times 0.5 \times 1 = 1.125$

Speedup of forwarding over pure stalling: For programs with 30% ALU hazards + 12.5% load hazards:

Without forwarding: CPI = 1 + 0.30×2 + 0.125×3 = 1.975
With forwarding: CPI = 1 + 0.125×1 = 1.125
Speedup = 1.975/1.125 = 1.76×

Common mistake "Forwarding Solves All Data Hazards"

Why it feels right: Forwarding seems like a complete solution—data is available, so why not always forward?

Why it's wrong:

Load-use hazards: Data doesn't exist until memory read completes (end of MEM stage), but next instruction needs it at start of EX stage (one stage earlier). Physics wins.
Structural hazards: If register file has only one write port, we can't forward to two instructions simultaneously in some designs.
Complexity limits: Full forwarding requires O(n²) forwarding paths for n pipeline stages, which becomes impractical for deep pipelines.

The fix: Recognize that forwarding is an optimization, not a panacea. Stalls are still necessary for fundamental timing constraints. Compiler scheduling is the second line of defense.

Connections

5.2.01-Five-stage-pipeline-basics - Foundation of the pipeline stages where hazards occur
5.2.07-Control-hazardsand-branch-prediction - Other major hazard type (control flow)
5.2.09-Pipeline-performance-analysis - How hazards impact overall CPI
5.3.02-Superscalar-out-of-order-execution - Advanced forwarding in modern processors
3.1.05-Register-file-design - Hardware that forwarding bypasses
6.2.03-Compiler-optimization-techniques - Instruction scheduling to minimize hazards

Recall Explain to a 12-Year-Old

Imagine you're building Lego spaceships on an assembly line. You have 5 stations: grab pieces (IF), check instructions (ID), snap pieces together (EX), add special parts from a box (MEM), and put the finished section on the shelf (WB).

Now, Station 3 is building a red wing, and Station 2 (right behind it) needs that same red wing to attach to the body. But the wing isn't on the shelf yet—it's still being built at Station 3!

Bad solution: Stop the whole line and wait until the wing reaches the shelf. Slow!

Smart solution (forwarding): Station 3 just hands the wing directly to Station 2. Why put it on the shelf first if the next person needs it right now?

But sometimes you need a piece from the special parts box (memory). That box is slow—you have to walk over, unlock it, and find the piece. Even with forwarding, you still have to wait one turn because the piece literally doesn't exist until you open the box. That's a load-use hazard.

Forwarding is like teamwork: pass the result directly to whoever needs it next instead of following every rule about putting stuff away first.

#flashcards/hardware

What is a data hazard in a pipeline? :: When an instruction needs a register value that a previous instruction hasn't written yet, because the previous instruction is still in the pipeline. The dependent instruction would read stale data.

What are the three types of data hazards?

RAW (Read After Write - true dependency), WAR (Write After Read - anti-dependency), WAW (Write After Write - output dependency). RAW is the most common in simple pipelines.

What is forwarding/bypassing?

A technique that routes computed results directly from pipeline registers (EX/MEM or MEM/WB) to the ALU inputs of dependent instructions, avoiding the need to wait for write-back to the register file.

What is a load-use hazard?

When an instruction immediately following a load needs the loaded value. Even with forwarding, one stall cycle is required because the data doesn't exist until the load's MEM stage completes.

What are the two main forwarding paths in a 5-stage pipeline?

EX/MEM to EX stage (for results one cycle old) and MEM/WB to EX stage (for results two cycles old). EX/MEM has priority when both match.

Why can't forwarding eliminate load-use hazards?

The loaded data doesn't exist until the end of the MEM stage, but the dependent instruction needs it at the start of the EX stage (one stage earlier). The data can't be forwarded before it exists.

How many stall cycles are needed for a load-use hazard with forwarding?

One stall cycle. Without forwarding, it would be two or three cycles depending on the architecture.

What is the forwarding unit?

Combinational logic that monitors pipeline registers, compares destination and source registers, and generates ForwardA/ForwardB control signals for ALU input multiplexers.

Why does EX/MEM forwarding have priority over MEM/WB forwarding?

Because EX/MEM contains the more recent result. If two instructions write the same register, the dependent instruction should use the newer value.

What does ForwardA = 10 mean?

Forward the ALU result from the EX/MEM pipeline register to ALU input A (the first operand). 10 is the encoding for EX/MEM forwarding.

What is instruction scheduling in the context of hazards?

A compiler optimization that reorders independent instructions to fill load delay slots and minimize pipeline stalls while preserving program semantics.

What condition creates an EX-to-EX forwarding opportunity?

When EX/MEM.RegWrite = 1, EX/MEM.RegisterRd ≠ 0, and EX/MEM.RegisterRd equals either ID/EX.RegisterRs or ID/EX.RegisterRt.

Why do we check RegisterRd ≠ 0 in forwarding logic?

Register 0 is hardwired to zero in architectures like MIPS and RISC-V, so writes to it don't create real dependencies. No forwarding is needed for R0.

What is a pipeline bubble?

A NOP (no operation) inserted into the pipeline to stall execution, effectively creating an empty pipeline stage where no useful work is done.

How does forwarding improve CPI?

By eliminating most ALU-to-ALU data hazard stalls. Typical improvement: from CPI ≈ 1.975 (stall-only) to CPI ≈ 1.125 (with forwarding) a 1.76× speedup.

Concept Map

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho yaar, pipeline ek assembly line jaisa hai jahan multiple instructions ek saath different stages mein chal rahe hote hain. Problem tab aati hai jab ek instruction ko previous instruction ka result chahiye, lekin wo result abhi tak register mein likha hi nahi gaya. Jaise agar ADD instruction R1 mein answer daal raha hai, aur uske turant baad SUB ko wahi R1 padhna hai, toh SUB apne ID stage mein pahunch jaata hai lekin ADD ne abhi tak WB (write back) kiya hi nahi hota. Isko hum data hazard kehte hain, aur sabse common type hai RAW (Read After Write) jahan ek instruction wahi cheez padhna chahta hai jo dusra abhi likh raha hai. Bina kisi fix ke, SUB purana ya stale data utha lega aur galat computation ho jaayega.

Ab iska ek seedha-saada solution hai stalling — matlab pipeline mein "bubbles" ya NOPs daal do taaki dependent instruction thoda wait kare jab tak data ready na ho jaaye. 5-stage pipeline mein ALU-to-ALU dependency ke liye tumhe roughly 2 cycles rukna padta hai, kyunki data tab tak register file mein properly write nahi hota jab tak WB stage complete na ho jaaye. Lekin dhyaan rakho — ye maan lena ki "sirf ek cycle stall karna kaafi hai" ek common galti hai, kyunki register file ka read write se pehle ho jaata hai clock edge par, isliye actual gap zyada hota hai. Ye samajhna zaroori hai warna tum stall count galat lagaoge.

Ye topic isliye matter karta hai kyunki stalling se performance kaafi gir jaati hai — 3 wasted cycles ka matlab hai teen guna slowdown sirf ek chhoti si instruction sequence ke liye. Real processors mein isiliye forwarding (ya bypassing) use hoti hai, jahan result ko directly EX stage se le liya jaata hai bina register file ka wait kiye. Agar tum data hazards ki root cause samajh loge, toh aage forwarding, pipeline optimization, aur modern CPU design ke concepts bahut aasaan lagenge. Ye foundation hai jispe pura efficient processor performance khada hota hai.

Data hazards and forwarding - bypassing

What Are Data Hazards?

Why RAW Hazards Dominate

Solution1: Stalling (The Naive Fix)

Solution 2: Forwarding/Bypassing (The Smart Fix)

Forwarding Paths: The Architecture

Load-Use Hazards: When Forwarding Isn't Enough

Forwarding Unit Implementation

Performance Impact Analysis

Connections

Concept Map

Hinglish (regional understanding)

Go deeper — visual, from zero

Test yourself — Processor Datapath & Pipelining

Connections