4.1.9Computer Architecture (Deep)

Cache organization — direct-mapped, n-way set associative, fully associative

2,376 words11 min readdifficulty · medium3 backlinks

WHY does cache need "organization" at all?


WHAT are the address fields? (Derivation from scratch)

We never memorize the split — we derive it.

Step 1 — Offset. To pick a byte inside a block of BB bytes we need b=log2B bits (block offset).b = \log_2 B \text{ bits (block offset)}. Why? BB distinct bytes require log2B\log_2 B bits to index them.

Step 2 — Set index. The block is mapped to a set by set=(block number)modS,s=log2S bits (index).\text{set} = (\text{block number}) \bmod S,\qquad s=\log_2 S \text{ bits (index)}. Why mod SS? We chop the address space into SS groups; identical low bits → same set. Using the low block-number bits spreads consecutive blocks across different sets (good for spatial locality).

Step 3 — Tag. The remaining high bits identify which of the many blocks that map to this set is actually stored: t=msb bits (tag).t = m - s - b \text{ bits (tag)}.


The three organizations as ONE formula

Let cache hold N=SEN = S\cdot E blocks total.

Organization EE (ways) SS (sets) Placement of block #kk
Direct-mapped E=1E=1 S=NS=N exactly slot kmodNk \bmod N
n-way set assoc. E=nE=n S=N/nS=N/n any of nn slots in set kmodSk \bmod S
Fully associative E=NE=N S=1S=1 any slot (no index bits!)
Figure — Cache organization — direct-mapped, n-way set associative, fully associative

HOW does a lookup work? (mechanism)

Number of tag comparators needed =E= E (the associativity). That's why fully-associative (E=NE=N) is hardware-expensive: NN comparators firing every access.


Worked examples


Three kinds of misses (the "3 C's")


Common mistakes (Steel-man + fix)


Active recall

Recall Why does increasing associativity increase tag size (fixed cache size)?

Fixed N=SEN=S\cdot E. More ways \Rightarrow fewer sets \Rightarrow fewer index bits \Rightarrow since t=msbt=m-s-b, tag grows.

Recall How many tag comparators does an

EE-way cache need? Exactly EE — one per line in a set, all compared in parallel.

Recall Which miss type does fully-associative eliminate, and which it cannot?

Eliminates conflict misses; compulsory and capacity still occur.

Recall (Feynman, explain to a 12-year-old)

Imagine a coat-check room. Direct-mapped: your coat number tells you the one exact hook it must hang on — fast, but if two coats share a number, one gets bumped. Fully associative: any hook is fine, so coats never get bumped, but the worker must look at every hook to find yours — slow. n-way: the room is split into small shelves; your number picks a shelf, and there are nn hooks on it — a couple of friends can share a shelf without bumping, and the worker only checks nn hooks. The tag is the name written on the coat so you know it's really yours.


Flashcards

What does cache organization decide?
Which slot(s) a memory block is allowed to occupy in the cache.
Address split, high to low bits?
Tag | Index | Offset.
Formula for block offset bits given block size B?
b=log2Bb=\log_2 B.
Formula for index bits given number of sets S?
s=log2Ss=\log_2 S.
Formula for tag bits?
t=msbt = m - s - b (address width minus index minus offset).
Direct-mapped in terms of (S,E)?
E=1E=1, S=NS=N (one block per set).
Fully associative in terms of (S,E)?
S=1S=1, E=NE=N, so index bits s=0s=0.
n-way set associative: how is the set chosen?
set=(block number)modS\text{set} = (\text{block number}) \bmod S, then any of the nn lines in it.
How many tag comparators for E-way?
Exactly EE, all in parallel.
For fixed cache size, what happens to tag bits as associativity rises?
Tag bits increase (sets/index decrease).
Which organization needs no replacement policy and why?
Direct-mapped — a block has only one legal slot, no choice.
Three C's of cache misses?
Compulsory (cold), Capacity, Conflict.
Which miss type does full associativity eliminate?
Conflict misses.
Why is fully associative expensive in hardware?
It needs NN comparators (one per line) active every access, plus largest tags.
A 32-bit, 16KiB, 64B-block, 4-way cache field split?
Tag 20 | Index 6 | Offset 6.
What must both match for a hit?
Valid bit = 1 AND stored tag = address tag.

Connections

Concept Map

is a

determines

contains

contains

contains

picks byte in block

selects set via mod S

identifies block in set

special case

special case

special case

max index min tag

no index bits

causes many

avoids

balances

Cache organization choice

Search cost vs flexibility

Address split t s b

Offset b = log2 B

Index s = log2 S

Tag t = m - s - b

Direct-mapped E=1

n-way set assoc E=n

Fully associative S=1

Conflict misses

Hit search hardware

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Cache organization ka matlab sirf ek sawaal hai: jab main memory ka koi block cache me laana hai, toh wo cache ke kaun se slot me ja sakta hai. Direct-mapped me block ka sirf ek hi fixed ghar hota hai (block number mod total slots) — bahut fast aur sasta, par agar do alag blocks ka same slot ban jaye toh wo ek-doosre ko baar-baar nikaalte rehte hain, isko conflict miss bolte hain. Fully associative me block kahin bhi ja sakta hai — koi conflict nahi, par har slot ko check karna padta hai, isliye hardware mehnga (har line pe ek comparator). n-way beech ka raasta hai: pehle index se ek set choose karo, fir us set ke andar n slots me se kisi ek me daal do.

Address ko hum teen hisson me todte hain: Tag | Index | Offset. Offset block ke andar ka byte choose karta hai (b = log2 BlockSize). Index set choose karta hai (s = log2 NumberOfSets). Tag baaki bache bits, jo batata hai us slot me actually kaunsa block pada hai. Yaad rakho: cache size fix ho aur associativity badhao, toh sets kam ho jate hain, index bits kam, aur tag bits badh jaate hain — yeh exam me sabse common trick hai.

Hit kab hota hai? Jab us set ki kisi line ka valid bit = 1 ho aur uska stored tag address ke tag se match kare. Direct-mapped me replacement policy ki zaroorat hi nahi, kyunki sirf ek hi jagah hai — choice hi nahi. Yeh saari cheezein bas ek hi design decision (SS aur EE choose karna) se nikalti hain, isliMumbaiE ratne ki zaroorat nahi, derive kar lo.

Go deeper — visual, from zero

Test yourself — Computer Architecture (Deep)

Connections