5.4.14Memory Hierarchy & Caches

Cache coherence problem

1,944 words9 min readdifficulty · medium

WHY does this problem even exist?

WHAT we want: every core should see memory as if there were one single, up-to-date copy of every address. This illusion is called a coherent memory system.

WHY it breaks: caches exist for speed. Each core keeps a private copy of hot data so it doesn't pay the slow trip to main memory every time. But the moment you have more than one private copy, a write to one copy does not automatically update the others. Now two cores can disagree about what lives at address X.


The concrete scenario

Figure — Cache coherence problem

Setup: Core A and Core B each have a private cache. Shared variable X = 5 in main memory.

Step Action Cache A Cache B Memory
0 initial X=5
1 A reads X X=5 X=5
2 B reads X X=5 X=5 X=5
3 A writes X=10 (write-back) X=10 X=5 ❌ X=5 ❌
4 B reads X X=10 X=5 (stale!) X=5

At step 4, B reads its own cached copy and gets 5, but the true logical value is 10. That is incoherence.


Defining coherence precisely


HOW do we detect there's a bug? (a mini "derivation")

We can derive the failure condition from first principles instead of memorising it.

Let a cached line be valid if cache_value == true_logical_value. A read returns the cache value when valid, else fetches from memory.

Define an invariant we WANT:  cores i,j:Validi(X)  Validj(X)  cachei(X)=cachej(X)\forall \text{ cores } i,j:\quad \text{Valid}_i(X)\ \wedge\ \text{Valid}_j(X)\ \Rightarrow\ \text{cache}_i(X)=\text{cache}_j(X)

Now consider a write on core ii: cache_i(X) := new. This assignment changes only ii's copy. Unless the protocol also does one of:

  • Validj(X):=false\text{Valid}_j(X) := \text{false} (invalidate other copies), or
  • cachej(X):=new\text{cache}_j(X) := new for all jj (update other copies),

the invariant is violated whenever another valid copy existed. This gives us the two protocol families:


Worked examples


Forecast-then-verify

Recall Answer

B reads stale 5 (from its own cache). Memory is also still 5, because write-back hasn't flushed A's dirty line yet. Two independent bugs: cache-vs-cache and memory-vs-cache. A protocol is required to fix the first; write-back timing controls the second.


Flashcards

What is the cache coherence problem?
When multiple private caches hold copies of the same address, a write to one copy can leave other caches (and/or memory) holding stale values, so cores disagree on the value.
Name the 3 conditions for coherence.
(1) A processor reads its own last write (program order), (2) write propagation — a write is eventually seen by others, (3) write serialization — all cores see writes to one location in the same order.
Does write-through solve coherence?
No. It keeps memory fresh but does not update or invalidate other caches; those still read their stale copies.
Coherence vs consistency?
Coherence = behaviour of a single address across caches; consistency = ordering of operations across different addresses.
Two ways to enforce coherence after a write?
Write-invalidate (mark other copies invalid) or write-update/broadcast (push new value to other copies).
Why is write-invalidate usually preferred over write-update?
Repeated writes to the same line generate only one invalidation but many update broadcasts, so invalidate produces less bus traffic.
When can write-update beat write-invalidate?
In tight producer/consumer sharing where the reader needs the value immediately after each write, update avoids a subsequent read miss.
What restores the coherence invariant after a write on core i?
Eliminate stale valid copies elsewhere — either invalidate them or update them with the new value.

Recall Feynman: explain to a 12-year-old

Imagine you and your friend each keep a personal notebook with your team's score copied from the big scoreboard. You score a point and change your notebook to a new number — but your friend's notebook and even the big board still show the old number. Now nobody agrees. To fix it, either you shout "cross out your number!" (invalidate) or "everyone change it to the new number!" (update). That's exactly what CPU caches must do so all cores agree on what's really stored.

Connections

  • Write-through vs Write-back — controls the memory-vs-cache gap (a different gap from coherence).
  • MESI Protocol — the standard write-invalidate state machine that solves this.
  • Snooping vs Directory Coherencehow invalidations are broadcast/tracked.
  • Memory Consistency Models — ordering across different addresses.
  • False Sharing — a performance side effect of line-granular coherence.
  • Bus Traffic & Scalability — why update protocols don't scale.

Concept Map

create

of same address

does not update

B reads old value

violates

updates memory only

still needs

invalidate or update

defined by

self order + propagation + serialization

distinct from

Caches for speed

Multiple private copies

Coherence problem

Core A writes X=10

Cache B stale copy

Coherent memory illusion

Write-through

Cache-vs-cache gap remains

Coherence protocol

3 conditions

Consistency across addresses

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho, cache coherence problem tab aati hai jab multiple cores ke paas apni-apni private cache hoti hai aur wo same memory address ki copy rakhte hain. Speed ke liye har core apni cache mein hot data ka photocopy rakh leta hai. Problem yeh hai ki jab ek core us value ko write karta hai, to sirf uski apni copy change hoti hai — baaki caches ke paas purani (stale) value padi rehti hai. Ab do cores ek hi address pe alag-alag value dekh rahe hain — yahi incoherence hai.

Ek common galatfehmi: "write-through use kar lo, memory fresh rahegi, problem khatam." Nahi bhai! Write-through sirf memory ko update karta hai, lekin doosri cache ki copy ko touch hi nahi karta. Wo cache abhi bhi apni purani value read karegi. Coherence ka masla cache-vs-cache ka hai, memory-vs-cache ka nahi.

Fix kya hai? Jab koi core write kare, to ya to baaki copies ko invalidate kar do (bolo: "tumhari copy ab galat hai, dobara fetch karo"), ya sab ko naya value update/broadcast kar do. Real CPUs (jaise MESI protocol) zyadatar invalidate use karte hain, kyunki agar ek core baar-baar same line pe likh raha hai to invalidate sirf ek message bhejta hai, jabki update har write pe broadcast karega — bahut traffic. Lekin producer-consumer jaise cases mein, jahan reader ko turant value chahiye, update better ho sakta hai. Matlab pattern dekh ke decide hota hai — koi ek universally best nahi.

Go deeper — visual, from zero

Test yourself — Memory Hierarchy & Caches

Connections