5.4.10
Intuition Ek-sentence wali ideaMemory zyaadatar time fast hoti hai (cache hits) lekin kabhi-kabhi bahut slow ho jaati hai (misses jo main memory tak pahunch jaate hain). AMAT woh weighted average time hai jo aap actually har access pe wait karte ho — yeh rare-but-expensive misses se dominate hoti hai, aur isliye hum miss rate ke peeche padte hain.
AMAT = ek memory access complete karne ka expected time, hits aur misses dono ko average karke.
AMAT = Hit Time + ( Miss Rate × Miss Penalty )
Hit Time : cache access karne ka aur hit hone par data deliver karne ka time (yeh aap hamesha pay karte ho).Miss Rate : un accesses ka fraction jo is cache level mein not found hote hain.Miss Penalty : miss hone par next level se block fetch karne ka extra time.
Hum chahte hain expected time per access . Ek single access le:
T hi t = 1 − m m kuch zyaada agar yeh miss kare (probability = m
Key modeling decision: Har access par hum pehle cache mein dekhte hain — toh hum hamesha T hi t jab miss hota hai tab hum lower memory ka extra trip pay karte hain, T p e na l t y
Toh do outcomes hain:
Outcome
Probability
Time
Hit
1 − m T hi t
Miss
m T hi t + T p e na l t y
Expected value (average ki definition):
AMAT = ( 1 − m ) T hi t + m ( T hi t + T p e na l t y ) Expand karo aur T hi t
= T hi t − m T hi t + m T hi t + m T p e na l t y = T hi t + m ⋅ T p e na l t y
T hi t koi miss-rate KYUN nahi hai
Aap hit time har ek access par pay karte ho (aap hamesha cache check karte ho), isliye yeh ek flat, unavoidable cost hai. Sirf penalty "kya hum miss hue?" se gated hai, isliye sirf usi ko m
L1 ki "miss penalty" khud L2 ka AMAT hai. Memory ek hierarchy hai, isliye hum formula ko nest karte hain:
AMAT = H T L 1 + m L 1 ( H T L 2 + m L 2 ⋅ M P L 2 ) jahan M P L 2
Intuition Layers KYUN help karte hainL2 add karna ek huge single miss penalty (jaise 200 cycles to DRAM) ko ek chhoti penalty (jaise 10 cycles to L2) se replace karta hai zyaadatar time. Ab rare L1-miss usually L2 mein hit hoti hai, toh expensive DRAM trip ko product m L 1 × m L 2
Worked example Single-level cacheDiya hai: H T = 1 m = 5% = 0.05 M P = 100 AMAT = 1 + 0.05 × 100 = 1 + 5 = 6 ns Yeh step kyun? Hum hamesha 1 ns hit pay karte hain; phir 5% accesses har ek 100 ns trip add karte hain, jo average mein 0.05 × 100 = 5 yahi AMAT ka pura point hai .
Worked example Two-level cacheH T L 1 = 1 m L 1 = 10% H T L 2 = 10 m L 2 = 20% M P L 2 = 200 H T L 2 + m L 2 ⋅ M P L 2 = 10 + 0.2 × 200 = 10 + 40 = 50 ns Kyun? Yahi hai jo ek L1 miss actually cost karta hai — effective L1 miss penalty.
Ab L1 mein plug karo:
AMAT = 1 + 0.10 × 50 = 1 + 5 = 6 ns Yeh step kyun? Scary 200 ns DRAM cost 0.10 × 0.20 = 0.02 0.02 × 200 = 4
Worked example Forecast-then-Verify: kya L2 worth it hai?Forecast: Bina L2 ke, ek L1 miss seedha DRAM jaati hai: AMAT = 1 + 0.10 × 200 = 21 Verify: L2 ke saath humne 6 ns compute ki. Toh L2 ne access time 21 → 6 ns kar diya (~3.5× faster). ✅ Forecast confirmed — layering ne average ko crush kar diya.
Common mistake Hit time ko hit rate se multiply karnaGalat idea: AMAT = ( 1 − m ) H T + m M P H T Kyun sahi lagta hai: "Hit par H T M P Fix: Ek miss mein phir bhi pehle cache check karna pada, toh aap H T phir M P + H T H T + m ⋅ M P
Common mistake Miss penalty ko total miss time se confuse karnaGalat: "time to reach DRAM = 200 ns" ko AMAT-on-miss including hit time ke roop mein use karna, phir hit time dobara add karna.
Fix: Miss penalty woh extra time hai jo hit access se aage hai. Consistently define karo: kya M P M P additional cost treat karta hai.
Common mistake Multi-level mein local vs global miss rate ki mix-upGalat: recursive formula ke andar global L2 miss rate use karna.
Fix: Har cache sirf wohi accesses dekhta hai jo uske paas forward hue → nested AMAT ke andar L2 ka local miss rate use karo. Global rate overall system stats compute karne ke liye hai, inner term ke liye nahi.
Recall Cover karo aur answer do
AMAT formula batao aur kyun H T
Do-outcome expected value se AMAT derive karo.
DRAM abhi bhi slow hone ke bawajood L2 add karne se AMAT kyun reduce hoti hai?
Local aur global miss rate mein kya fark hai?
Example 1 mein AMAT ka kitna fraction misses se aaya?
Recall Feynman: ek 12-saal ke bacche ko explain karoSocho tumhara school locker (fast) zyaadatar books hold karta hai. Zyaadatar baar tum instantly ek book uthate ho — yeh ek hit hai. Lekin kabhi-kabhi book door ki library mein hoti hai (slow) — yeh ek miss hai. Average mein, ek book lene mein kitna time lagta hai? Almost hamesha fast, lekin rare library trips itni slow hoti hain ki woh average ko upar khench deti hain. AMAT bas wahi average waiting time hai. Use chhota karne ke liye hum locker ko faster banane ki koshish nahi karte — hum library trips ko rarer banane ki koshish karte hain (fewer misses) ya ek nearby "shelf" (L2) add karte hain taaki hum rarely library tak jaayein.
"HiT Me Poorly" → H it time + M iss rate × M iss P enalty.
Aur: "Tum hamesha door se Hit karte ho; sirf Missers toll pay karte hain."
AMAT formula (single level) AMAT = H T + MissRate × MissPenalty
Hit time ko miss rate se kyun multiply nahi kiya jaata? Aap har request par cache access karte ho, toh aap hamesha H T
Expected value se AMAT derive karo ( 1 − m ) H T + m ( H T + M P ) = H T + m M P
Two-level AMAT formula H T L 1 + m L 1 ( H T L 2 + m L 2 M P L 2 )
L2 ka local miss rate L2 misses ÷ accesses jo L2 tak pahunche .
L2 ka global miss rate L2 misses ÷ saare CPU accesses = m L 1 × m L 2 , l oc a l
Recursive AMAT term ke andar kaun sa miss rate jaata hai? Us level ka local miss rate.
Miss penalty ki definition Hit access se aage next level se block fetch karne ka extra time.
Single-level: HT=1ns, m=5%, MP=100ns → AMAT? 1 + 0.05 × 100 = 6
L2 cache kyun add karo agar DRAM abhi bhi slow hai? L1 misses mostly L2 mein hit hoti hain, toh huge DRAM penalty tiny product m L 1 × m L 2
Cache Miss Rate & Miss Types (3 Cs) Miss Penalty & Main Memory Latency Multi-level Cache Hierarchy Cache Associativity & Hit Time tradeoff CPU Performance Equation (AMAT memory-stall cycles mein feed hoti hai)Locality of Reference (hit rates high kyun hoti hain)
reduces expensive DRAM trips
AMAT expected access time
Expected value over hit and miss
Multi-level recursive AMAT
