5.4.16 · D4Memory Hierarchy & Caches

Exercises — Memory consistency models

2,055 words9 min readBack to topic

Before we start, one shared vocabulary reminder so no symbol is unearned:

The reordering table we keep reusing (from the parent):

Model L→L L→S S→S S→L
SC
TSO (x86)
PSO
Weak (ARM/POWER)

(✅ = the model keeps them in order; ❌ = the model may swap them.)

Figure — Memory consistency models

Level 1 — Recognition

L1.1

Which property — coherence or consistency — is about a single memory address, and which is about ordering across different addresses?

Recall Solution
  • Coherence = single address. All cores must agree on the order of writes to one cell. See Cache Coherence.
  • Consistency = across addresses. It governs how ops to different cells are ordered relative to one another. Coherence is necessary but weaker; consistency is the global rulebook.

L1.2

On plain x86 (TSO), which one of the four program-order pairs is allowed to be reordered?

Recall Solution

Only S→L (an earlier Store may be bypassed by a later Load). Reason: the store sits in the per-core FIFO store buffer and drains later, while the load completes now. Mnemonic: "Stores Look Late."

L1.3

True or false: volatile in C/C++ inserts hardware fences and gives cross-thread ordering.

Recall Solution

False. volatile only stops the compiler from caching a value in a register; it emits no hardware fence and gives no cross-thread ordering. For that you need std::atomic with an ordering (see C++ Memory Model (acquire-release)).


Level 2 — Application

L2.1 — Store Buffering (SB)

Init x = 0, y = 0.

Core 1 Core 2
x = 1 (S1) y = 1 (S3)
r1 = y (L2) r2 = x (L4)

Can r1 == 0 && r2 == 0 under (a) SC, (b) TSO?

Recall Solution
  • (a) SC: No. SC enforces every pair, including S→L. So S1 L2 and S3 L4. r1==0 needs L2 S3; r2==0 needs L4 S1. Chain them: — a cycle, impossible. Forbidden.
  • (b) TSO: Yes. TSO drops S→L. Both stores sit in store buffers; both loads read the stale 0 from memory before either store drains. Outcome r1==r2==0 is legal on x86.

L2.2 — Message passing on x86

Init data = 0, flag = 0.

Core 1 (producer) Core 2 (consumer)
data = 42 (S1) while(flag==0){} (L3)
flag = 1 (S2) r = data (L4)

On TSO/x86, is a fence needed to guarantee r == 42?

Recall Solution

No fence needed on TSO. The producer's two stores are S→S, which TSO preserves, so data=42 becomes globally visible before flag=1. The consumer's L3→L4 is L→L, also preserved. Once L3 sees flag==1, data is already 42. (On weak hardware this would need fences — see L3.2.)


Level 3 — Analysis

L3.1 — Cycle test on a custom outcome

Same SB test as L2.1, but the target outcome is r1 == 1 && r2 == 0 under SC. Is it legal? Build the ordering graph.

Recall Solution

Legal. Edges forced:

  • Program order: , .
  • r1==1 (L2 read the new y) ⇒ .
  • r2==0 (L4 read the old x) ⇒ . Now try to find a cycle. One valid total order: . Check: ✅ (po), ✅ (po), ✅, ✅. No cycle ⇒ allowed. A single acyclic ordering witnessing all edges = a legal SC execution.

L3.2 — IRIW under weak memory

Init x = y = 0. C1: x=1. C2: y=1. C3: r1=x; r2=y. C4: r3=y; r4=x. Can we observe r1=1, r2=0, r3=1, r4=0 on (a) SC/TSO, (b) POWER/ARM?

Recall Solution

This outcome means C3 sees x-write first, C4 sees y-write first — the two writes observed in opposite orders.

  • (a) SC/TSO: No. Both guarantee a single total store order. If in that one order precedes , every reader that sees must also see . C4 seeing r3=1,r4=0 contradicts C3 seeing r1=1,r2=0.
  • (b) POWER/ARM: Yes. These are non-multi-copy-atomic: a write can reach different cores at different times. No global store order is imposed, so the two readers legally disagree. (See Out-of-Order Execution for why propagation is not simultaneous.)

Level 4 — Synthesis

L4.1 — Fix the flag pattern for ARM

Take L2.2's message-passing code. Insert the minimum fences so r == 42 is guaranteed on weak (ARM) hardware. Name each fence's role.

Recall Solution

Two fences, one per core:

  • Producer — a release fence between S1 and S2: data = 42; RELEASE_FENCE; flag = 1; Role: forces S1 (data) to be globally visible before S2 (flag). Recovers S→S at exactly the point that matters.
  • Consumer — an acquire fence after L3: while(flag==0){}; ACQUIRE_FENCE; r = data; Role: forbids L4 (read data) from floating above L3 (read flag). Recovers L→L. Together they form an acquire–release pair (C++ Memory Model (acquire-release)): everything before the release on Core 1 is visible after the matching acquire on Core 2. Cheaper than making the whole program SC.

L4.2 — Choose the cheapest tool

You need r1==0 && r2==0 to be impossible in the SB test (L2.1) running on x86. What is the minimal change?

Recall Solution

Insert one fence per core between the store and the load: x = 1; MFENCE; r1 = y; and y = 1; MFENCE; r2 = x; MFENCE (or any LOCK-prefixed atomic) drains the store buffer, restoring the S→L ordering TSO dropped. Now S1 L2 and S3 L4 hold ⇒ the same cycle argument as SC forbids the double-zero. No coherence change and no lock is needed — just the two fences.


Level 5 — Mastery

L5.1 — Design & judge a hand-off protocol

Design a single-producer/single-consumer one-slot buffer usable on weak hardware, then prove the consumer never reads stale data. State each ordering guarantee you rely on and where it comes from.

Recall Solution

Protocol (uses one full flag as the acquire-release handshake):

Producer:            Consumer:
  buf = item           while(load_acquire(full)==0){}
  store_release(full,1)  x = buf
                         store_release(full,0)   // reclaim slot

Proof the consumer never sees stale buf:

  1. store_release(full,1) guarantees every write before it (here buf=item) is visible to any core that performs the matching load_acquire(full) and sees the value 1. — release semantics.
  2. The consumer's load_acquire(full) observing 1 therefore synchronizes-with the producer's release. — acquire semantics.
  3. Consequently x = buf (which is after the acquire in program order, and acquire forbids later loads from hoisting above it) reads the value the producer wrote. ∎
  4. Coherence (single-address agreement, Cache Coherence) guarantees the full flag itself is seen consistently, so the spin-wait terminates once. Consistency handles the cross-address ordering between full and buf.

Judgment: correct and cheap — exactly one acquire and one release fence on the hot path, nothing global. This is strictly weaker (thus faster) than making the whole region SC, yet sufficient.

L5.2 — Falsify a claim

A colleague claims: "On x86 I never need any fence, because x86 is TSO and TSO is 'basically SC'." Give the one program that falsifies this, and the fence that saves it.

Recall Solution

Falsifier = the SB test (L2.1). TSO's single relaxation, S→L, makes r1==0 && r2==0 observable on real x86 — an outcome SC forbids. So TSO is not SC; the difference is exactly the store-buffer bypass. Fix: MFENCE between each core's store and load (as in L4.2) drains the buffer and forbids the double-zero. Conclusion: "TSO ≈ SC" is false precisely for the S→L pair, and that pair is not an academic curiosity — it is the everyday Dekker/mutex-flag hazard.


Recall One-line self-check

The universal legality test for any outcome ::: build the memory-order graph from program-order + read-from edges, then check for a cycle; a cycle means the outcome is forbidden.