5.4.16Memory Hierarchy & Caches

Memory consistency models

2,381 words11 min readdifficulty · medium6 backlinks

WHAT is a memory consistency model?

Two things it is NOT, so we don't confuse them:


WHY do we even need one?

The canonical litmus test (Dekker / Store Buffering)

Initial: x = 0, y = 0.

Core 1 Core 2
x = 1 (S1) y = 1 (S3)
r1 = y (L2) r2 = x (L4)

Question (Forecast-then-Verify — predict before reading!): Can r1 == 0 && r2 == 0?

  • Under a strong model (Sequential Consistency): No. Some store must be globally first.
  • Under a weak model (TSO / x86 with store buffers): Yes — both stores sit in store buffers, both loads read stale 0 from memory.

The spectrum from strong to weak

Figure — Memory consistency models

HOW do we reason formally? The "reorderings allowed" table

Think of each model as: which of the four orderings between two same-core memory ops does it enforce?

Model L→L L→S S→S S→L Notes
SC all preserved
TSO store buffer bypass
PSO + relaxed S→S
Weak/ARM need fences

(✅ = ordering enforced; ❌ = may be reordered.)

Deriving why TSO allows only S→L (from first principles)

  1. A store SS enters the FIFO store buffer and retires from the pipeline; it drains to memory later. → the core proceeds.
  2. A subsequent load LL checks the buffer (store-forwarding) then goes to cache. It can complete while SS is still buffered. → LL finishes before SS is globally visibleS→L reordered. ✅ explains the SB outcome.
  3. Stores are a FIFO buffer ⇒ they drain in program order ⇒ S→S preserved.
  4. Loads are not buffered this way and are ordered relative to each other and to earlier stores in the buffer ⇒ L→L, L→S preserved.

That's the whole TSO rulebook, derived — no memorization needed.


Restoring order: fences


Worked examples


Recall Feynman: explain it to a 12-year-old

Imagine four friends passing notes in class. Each friend writes their notes in order, but the notes travel around the room at different speeds. So Anna might get Bob's second note before his first! A memory consistency model is the class rule that says how out-of-order the notes are allowed to arrive. A strict rule says everyone must read all notes in the exact same order (slow but simple). A relaxed rule lets notes fly around fast, but if you really need someone to see two of your notes in order, you shout "FENCE!" and wait until the first note is delivered before sending the second.


Common mistakes


Flashcards

What is a memory consistency model?
A contract specifying which global orderings of loads/stores across processors are legal — defines allowed outcomes of a shared-memory program.
Consistency vs coherence?
Coherence orders writes to a single address; consistency orders operations across different addresses.
Statement of Sequential Consistency (SC)?
Result equals some single total interleaving of all ops in which each processor's ops appear in program order.
Which single reordering does TSO allow, and why?
Store→Load, because a store sits in a FIFO store buffer while a later load bypasses it via cache/store-forwarding.
Can r10 && r20 in the Store-Buffering test under SC?
No — it requires an ordering cycle. Under TSO/x86: Yes (store buffers).
What does a memory fence do?
Forces all memory ops before it to become globally visible before any op after it; e.g. MFENCE drains the store buffer.
Why doesn't C volatile give thread ordering?
It stops compiler register-caching but emits no hardware fences; use std::atomic with acquire/release/seq_cst.
What is IRIW and what distinguishes weak models on it?
Independent Reads of Independent Writes; multi-copy-atomic models (SC/TSO) forbid two cores seeing two writes in different orders, non-multi-copy (POWER/ARM) allow it.
General test for whether an outcome is legal under SC?
Build the required ordering edges; if they form a cycle, the outcome is illegal.

Connections

Concept Map

creates need for

is a

defines legal

distinct from

caused by

caused by

caused by

strong end

weaker end

enables outcome in

forbids r1=0 and r2=0

allows r1=0 and r2=0

Memory consistency model

Multiple cores + shared memory

Contract HW-programmer

Coherence per one address

HW reordering for speed

Store buffers

Out-of-order / interconnect

Compiler reordering

Sequential Consistency

Total Store Order x86

Store Buffering test

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho, jab ek hi memory ko multiple cores ek saath read/write karte hain, tab hardware speed ke liye operations ko reorder, buffer aur delay karta hai. Har core apne aap mein toh sahi chalta hai, lekin doosre cores ko memory operations alag order mein dikh sakte hain. Memory consistency model ek contract hai jo batata hai ki kaun-kaun se orderings allowed hain. Iske bina "yeh load kaunsi value return karega?" ka koi pakka jawab hi nahi hota.

Sabse strong model hai Sequential Consistency (SC) — socho ek single global switch hai jo baari-baari cores ke ops chalata hai, par kisi core ka apna program order kabhi todta nahi. Simple hai par slow. TSO (x86) thoda relaxed hai: ek store apne store buffer mein baith jaata hai aur core aage badh jaata hai, isliye baad ka load usse bypass kar sakta hai (sirf S→L reorder allowed). ARM/POWER toh bahut weak hain — almost sab kuch reorder ho sakta hai.

Yaad rakhne ka tareeka: TSO ka "one sin" — Stores Look Late (S→L). Jab tumhe cross-core ordering chahiye hi chahiye (lock, flag pattern), tab tum fence/barrier lagate ho jo store buffer ko drain kar deta hai aur ordering wapas la deta hai. Yeh 80/20 rule hai — sirf synchronization points par fence lagao, baaki 80% code fast chalne do.

Important trap: C ka volatile thread ordering nahi deta — woh sirf compiler ko register mein cache karne se rokta hai. Real ordering ke liye std::atomic (acquire/release/seq_cst) use karo. Aur yeh mat socho ki "x86 pe chal gaya matlab sab jagah chalega" — mobile ka ARM weak hai, wahan missing fence bug de dega.

Go deeper — visual, from zero

Test yourself — Memory Hierarchy & Caches

Connections