NUMA architectures: Question bank — Hardware deep dive

Before we start, one word we lean on repeatedly: a node is a bundle of "some cores + one local memory bank + one memory controller". "Local" means inside the same node as the running core; "remote" means in a different node, reached over the interconnect. Hold that picture — every trap below hinges on it.

The picture behind every trap

Look at the map below before you read a single question. Three ideas are drawn here that the rest of the page keeps referring to:

Left panel — four nodes wired by an interconnect. A local access (teal) stays inside one node; a remote access (orange) crosses one or more hops (each link you traverse is one hop). Node 3 is reached from node 0 by going through node 1: that is 2 hops, which is why it is slower.
The little latency ladder is the concrete measurement this page assumes throughout (a 4-node machine, 8 cores + 16 GB per node): node 0 = 85 ns (1.0x), node 1 = 140 ns (1.65x, 1 hop), node 2 = 165 ns (1.94x, 2 hops), node 3 = 168 ns (1.98x, 2 hops). Notice nodes 2 and 3 are almost equal — that is the trap in SE4.

Two more pictures we will point back to:

This is cache-line ping-pong (false sharing). Two threads on two different nodes keep writing different words that happen to sit in the same cache line. Every write drags the whole line across the interconnect — the arrow bounces back and forth, each bounce costing hundreds of nanoseconds.

This is a picture of the average-latency formula. It answers "if a fraction $f$ of my accesses are remote, what latency do I feel on average?" We literally split a bar of accesses into a local part and a remote part:

$t_{a v g} = the local slice (1 - f) t_{l oc a l} + the remote slice f t_{r e m o t e} .$

Now substitute the NUMA factor $F_{N U M A} = t_{r e m o t e} / t_{l oc a l}$ , i.e. $t_{r e m o t e} = F_{N U M A} t_{l oc a l}$ , and factor out $t_{l oc a l}$ :

$t_{a v g} = t_{l oc a l} [(1 - f) + f F_{N U M A}] .$

That bracket is just a weighted average of "1" (cost of a local access, in units of $t_{l oc a l}$ ) and " $F_{N U M A}$ " (cost of a remote one). When $f = 0$ the bracket is $1$ (all local); when $f = 1$ it is $F_{N U M A}$ (all remote). Everything in between slides along the tilted line in the figure. That is why the formula has exactly this shape — it is a straight-line blend controlled by the remote fraction $f$ .

True or false — justify

TF1. "In a NUMA system a processor cannot read memory that belongs to another node."

False. It absolutely can — NUMA gives a single shared address space. It just pays a higher latency (the remote path over the interconnect). The "N" in NUMA is Non-uniform, not No.

TF2. "UMA (uniform access) is always worse than NUMA."

False. For small core counts (≈4–8) UMA is often simpler and faster because there is no interconnect hop at all — every access is one flat cost. NUMA only wins once the single shared bus becomes the bandwidth bottleneck.

TF3. "ccNUMA means you must write message-passing code to talk between nodes."

False. ccNUMA hardware keeps caches coherent for you, so ordinary shared-memory loads and stores work across nodes. You need message-passing on non-coherent clusters, not on ccNUMA.

TF4. "Two threads on different nodes writing the same variable and two threads writing different variables in the same cache line cause the same kind of slowdown."

True in mechanism, subtle in cause. Both ping-pong the cache line across the interconnect (the bouncing arrow in the figure above). The first is true sharing (unavoidable if they truly share data); the second is false sharing (an accident of layout that padding can fix).

TF5. "Interleaved address mapping is better for locality than range-based mapping."

False. Interleaving scatters consecutive addresses across all nodes to maximize aggregate bandwidth; that deliberately destroys locality. Range-based keeps a contiguous block on one node, which is what lets the OS give a thread local memory.

TF6. "First-touch allocation happens when memory is malloc'd."

False. malloc (or the OS reserving the address range) does not pick a node. The page is physically placed on the node of whichever core first writes it. That is the whole point of the name "first-touch".

TF7. "The NUMA factor

F_{N U M A} = t_{r e m o t e} / t_{l oc a l}

is a property of the hardware alone."

True — it describes the latency ratio of the machine. But your achieved average latency also depends on

f

, the fraction of accesses that are remote, which is a property of your software.

TF8. "If 100% of accesses were remote, NUMA would still beat UMA."

Depends, and usually false for latency. With

f = 1

, average latency

= t_{l oc a l} \cdot F_{N U M A}

— strictly worse than a local access. NUMA's bandwidth advantage might still help under heavy load, but a purely remote workload throws away NUMA's main benefit.

TF9. "Cores inside one node see non-uniform latency to that node's own memory."

False. Within a single node the cores share the same controller and see uniform latency to local memory — a node is effectively a mini-SMP. The non-uniformity appears only between nodes.

TF10. "Once a page is placed by first-touch, it stays on that node forever."

False. Modern OSes can migrate pages between nodes at runtime (e.g. Linux AutoNUMA), moving a hot page to the node whose threads keep touching it. First-touch sets the initial placement, not a permanent one.

Spot the error

SE1. "To make code NUMA-friendly, just pin every thread to node 0."

Error: pinning all threads to one node crams every core's traffic through one memory controller — you have recreated the UMA bottleneck and starved the other nodes' bandwidth. Spread threads across nodes and put each thread's data on its own node.

SE2. "We initialized the whole array in main() before spawning worker threads, so it's ready to go."

Error: main() is one thread on one node, so first-touch places the entire array on that single node. Every worker on other nodes then does remote accesses. Initialize in parallel, letting each thread first-touch the region it will later use.

SE3. "Remote latency = local latency + interconnect latency, and that's the whole cost."

Error: it omits the remote controller time and the data return trip, and — for a coherent miss on modified data — the extra hops to snoop/writeback from a third node. A dirty remote line can be a 3-hop protocol, far more than one interconnect crossing.

SE4. "Node 2 is farther than node 1, so accessing node 3 must be even slower than node 2."

Error: latency tracks topology (hop count), not node numbering. On the machine drawn at the top of this page nodes 2 and 3 are both 2 hops away from node 0 — measured at 165 ns and 168 ns, essentially equal — while node 1 is 1 hop (140 ns). Never assume higher ID = farther; read the actual interconnect graph.

SE5. "Cache coherence is disabled in NUMA to make it fast, that's why it's called Non-uniform."

Error: two unrelated ideas mashed together. "Non-uniform" refers to latency varying by location. Coherence is a separate choice: ccNUMA keeps it on (in hardware); the name has nothing to do with turning coherence off.

SE6. "Since a directory-based protocol has a home node per line, a read is always a single request to the home node."

Error: if the line is modified in some other node's cache, the home directory must forward to that owner, wait for writeback/forward, then reply — the classic 3-hop path. Only clean lines resolve in one round trip.

SE7. "We got 45 GB/s after adding numactl --membind, so --membind alone is the fix."

Error: --membind forces where memory lives but not where threads run; you also need --cpunodebind (or first-touch) so the thread and its data land on the same node. Binding memory to node 0 while the thread runs on node 3 just guarantees remote access.

SE8. "Page migration is free real-estate — just turn on automatic NUMA balancing and forget tuning."

Error: migrating a page means copying it across the interconnect and updating page tables/flushing TLBs; if a page's access pattern flip-flops between nodes, the OS can thrash, migrating it back and forth and paying the cost repeatedly. Migration helps stable patterns, not ping-ponging ones.

Why questions

WHY1. "Why does a single shared bus physically stop scaling past ~4–8 processors?"

Every processor must arbitrate for the same electrical bus, so requests serialize; adding processors raises contention while the bus's fixed bandwidth is now split more ways. Electrically, more taps also degrade signal integrity and clock speed.

WHY2. "Why does the OS prefer range-based mapping over interleaving for general workloads?"

Range-based keeps a node's memory contiguous, so the OS can hand a thread a block from its own node (affinity). That enables first-touch and locality — the 80/20 win — which interleaving would destroy by scattering pages everywhere.

WHY3. "Why is false sharing so much worse on NUMA than on a single-socket UMA machine?"

The ping-ponged cache line now bounces across the interconnect (hundreds of ns per bounce, as in the ping-pong figure) instead of within one on-die coherence fabric (tens of ns). The same coherence pattern that was tolerable becomes a severe remote-traffic storm.

WHY4. "Why does the 'last 5%' of remote accesses still cost real performance?"

Average latency is

t_{l oc a l} [(1 - f) + f F_{N U M A}]

; at

f = 0.05, F_{N U M A} = 2

the bracket is

1.05

, i.e. a 5% latency tax, and speedup vs all-remote is ~1.9x not 2.0x. Remote accesses are expensive per event, so even a small fraction leaves a measurable gap.

WHY5. "Why can pushing locality from 80% to 95% give a smaller extra speedup than going 0% to 80%?"

Because speedup

\frac{F _{N U M A}}{( 1 - f ) + f F _{N U M A}}

is a curve with diminishing returns — most of the benefit is captured early (the first big drop in remote fraction), and the tail flattens as

f \to 0

. See parallel algorithms for the analogous Amdahl-style flattening.

WHY6. "Why does thread scheduling interact with NUMA at all?"

If the scheduler migrates a thread to a different node after its data was first-touched on the original node, all that data suddenly becomes remote. NUMA-aware scheduling tries to keep a thread on the node holding its working set.

WHY7. "Why does NUMA not automatically fix a memory-consistency bug in my code?"

NUMA changes how fast an access is, not what ordering guarantees exist. Ordering is governed by the memory consistency model and your synchronization — you still need the right fences/locks regardless of node placement.

WHY8. "Why would the OS migrate a page instead of migrating the thread?"

Sometimes the thread is pinned (by the scheduler or the user), or many threads share the page — moving the one page to where the accesses concentrate is cheaper than relocating threads and their whole working sets. It is the reverse lever for the same locality goal.

Edge cases

EC1. "Single-node NUMA system (or NUMA disabled in BIOS): what is

F_{N U M A}

With only one node every access is local, so

t_{r e m o t e} = t_{l oc a l}

and

F_{N U M A} = 1

. The performance model degenerates to plain UMA — the machine behaves uniformly.

EC2. "What if

f = 0

(perfect locality)?"

Average latency

= t_{l oc a l} [(1 - 0) + 0] = t_{l oc a l}

, the ideal. Speedup vs all-remote

= F_{N U M A}

, the full advantage the hardware can offer — this is the target every NUMA-aware program chases.

EC3. "The parent note maps an address to a node by

⌊ A / L ⌋ mod N_{n o d es}

, where

A

is the physical address (a byte offset into memory),

L

is the cache-line size in bytes, and

N_{n o d es}

is the number of nodes. What if

N_{n o d es} = 1

Then

⌊ A / L ⌋ mod 1 = 0

for every address, so all memory lives on node 0. Cores on other nodes are always remote — bandwidth collapses to one controller. (The

⌊ \cdot ⌋

is the floor, "round down to the nearest whole number";

mod

is the remainder after dividing.)

EC4. "A thread that touches a huge array once, sequentially, then never again — does first-touch still help?"

Only marginally. First-touch places pages locally for that thread, but if the data is read once with no reuse, there's little locality to exploit and streaming bandwidth dominates. Locality tuning pays off most when data is revisited.

EC5. "Two nodes, but the interconnect has 0 latency (idealized). What happens to the NUMA distinction?"

t_{in t er co nn ec t} \to 0

and the remote controller cost also vanished,

t_{r e m o t e} \to t_{l oc a l}

and

F_{N U M A} \to 1

— the machine looks uniform. Real interconnects are non-zero, which is exactly why the non-uniformity exists.

EC6. "All threads share one read-only lookup table. Best placement?"

Since it's read-only and shared by everyone, replicate a copy on each node (page replication) so every thread reads locally, or interleave it if replication isn't available. First-touch on one node would make it remote for all others.

EC7. "

F_{N U M A} < 1

— is that possible?"

No. Remote access adds an interconnect hop and a second controller on top of the local path, so

t_{r e m o t e} > t_{l oc a l}

always, giving

F_{N U M A} > 1

. A ratio below 1 would mean crossing the network is faster than staying home, which the physics forbids.

EC8. "A thread's working set was first-touched on node 0, but the scheduler later parks the thread on node 2 for good. The OS then migrates the pages to node 2. Was first-touch wasted?"

Not wasted — it was correct at the time; page migration is the correction when the steady-state access pattern shifts. The cost is a one-time copy across the interconnect; afterwards accesses are local again. The danger is only if the thread keeps bouncing (see SE8), which makes migration thrash.

Recall Quick self-test

The average-latency formula $t_{a v g} = t_{l oc a l} [(1 - f) + f F_{N U M A}]$ — what do $f$ and $F_{N U M A}$ each depend on? ::: $f$ (remote fraction) depends on your software (placement, first-touch, scheduling); $F_{N U M A}$ (latency ratio) depends on the hardware topology. Name the two ingredients you need for locality, and why binding only one fails. ::: You need the thread pinned to a node and its data first-touched/bound to that same node; binding only one leaves the pairing mismatched and every access remote. First-touch vs page migration — what does each control? ::: First-touch sets the initial node placement of a page; page migration moves an already-placed page later when the OS sees the access pattern has shifted to another node.