Average memory access time (AMAT): Foundations — Hinglish deep dive

5.4.10 · D1 · Hardware › Memory Hierarchy & Caches › Average memory access time (AMAT)

Parent note ka formula $AMAT = H T + m \cdot M P$ padhne se pehle, tumhe har letter ko point karke bolna aana chahiye ki woh kya hai. Yahi is page ka ek kaam hai. Hum inhe order mein build karte hain — har ek apne pehle wale pe lean karta hai.

0. Jo pictures hum use kar rahe hain

Neeche jo kuch bhi hai woh ek mental image pe jeeta hai: ek CPU (brain) jo baar baar poochh raha hai "mujhe is address ka data do," aur memories ka ek stack jo neeche jaate-jaate bada lekin slow hota jaata hai.

Upar di gayi ladder dekho. CPU sabse upar hai. Har rung ek jagah hai jahan data mil sakta hai: top ke paas ek chhota fast cache, neeche ek bada aur slow wala, phir sabse neeche bada slow main memory (DRAM). Har symbol jo hum define karte hain, woh is ladder pe kahin na kahin koi stopwatch reading ya counting-fraction hai. Poori ladder ki story ke liye Multi-level Cache Hierarchy dekho.

1. Memory access — woh event jise hum time kar rahe hain

Picture: ek arrow CPU se nikalti hai, ladder ke neeche jaati hai jab tak data milta nahi, phir wapas upar aati hai. Us round trip ki length (time mein) — yahi humein care karni hai.

Topic ko yeh kyun chahiye: AMAT ek per-access average hai. Agar hum yeh fix nahi karte ki "ek access" kya hota hai, toh averaging meaningless ho jaati. Yeh hamara counting unit hai. (Locality of Reference explain karta hai ki kyun consecutive accesses same fast rung pe land karte hain — yahi average ko low rakhta hai.)

2. Cache, hit, aur miss — kya humne data upar hi paaya?

Picture (upar): do coloured paths. Green path (hit) chhota hai — CPU cache se poochhhta hai, jawab milta hai, khatam. Red path (miss) lamba hai — cache ne haath khade kar diye, toh request slow memory tak neeche aur wapas jaati hai.

Topic ko yeh kyun chahiye: yahi do outcomes hain jo ek access pe ho sakte hain. AMAT literally inhi do cases ka average hai. Miss types (kyun misses hote hain) Cache Miss Rate & Miss Types (3 Cs) mein catalogued hain.

3. Time symbols — teen stopwatches

Ab hum un paths pe numbers lagate hain. Har ek ek duration hai, nanoseconds (ns) ya CPU cycles mein measure hua.

Picture: miss path pe, pehla segment (cache check) ki length $H T$ hai; journey ka baaki hissa $M P$ ki length hai. Total miss time $= H T + M P$ .

Topic ko yeh kyun chahiye: poora formula inhi do durations se bana hai. Yeh reason ki $H T$ final formula mein akele (un-multiplied) appear karta hai, woh yeh hai ki tum har access pe — hit ho ya miss — green check pay karte ho — yeh unavoidable hai.

4. Miss Rate — slow case kitni baar hota hai, us ka fraction

Average nikalne ke liye humein pata hona chahiye ki slow case kitni baar hota hai. Yeh ek fraction hai.

Picture (upar): 100 accesses ki ek bar. Ek patli red slice misses hain ( $m$ ), bada green hissa hits hain ( $1 - m$ ). Agar $m = 0.05$ hai, toh 100 mein sirf 5 requests long path lete hain — lekin jaise AMAT dikhayega, woh 5 tumhare wait time pe dominate kar sakte hain.

Topic ko yeh kyun chahiye: $m$ slow case ka weight hai. $M P$ ko $m$ se multiply karne par "penalty agar miss ho" se "penalty jo average per access pay hoti hai" ban jaati hai.

5. Average / Expected value — woh machine jo sab combine karti hai

Poore topic ke peeche yahi ek maths hai. AMAT ka matlab hai average memory access time, toh humें "average" ka matlab crystal clear hona chahiye.

Picture: 100 coins imagine karo, $p \times 100$ pe " $a$ " stamped hai aur $q \times 100$ pe " $b$ ." Saare stamps add karo aur 100 se divide karo — yahi average hai. Common outcomes average ko apni value ki taraf kheenchte hain; rare-but-huge outcomes phir bhi use noticeably upar kheench sakte hain.

Yeh sahi tool kyun hai (aur, say, sirf $(a + b) /2$ kyun nahi): ek plain midpoint pretend karega ki hits aur misses equally often hote hain. Nahi hote — hits common hain, misses rare. Weighted average un frequencies ko respect karta hai, jo "average mein kitna wait karunga" kehne ka ek maatra honest tarika hai.

6. Subscripts aur levels — $H T_{L 1}$ , $m_{L 2}$ padhna

Parent note caches stack karta hai (pehle L1, phir L2). Subscripts bas batate hain ladder ka kaunsa rung.

Picture: ladder se neeche paani beh raha hai. Saara paani L1 se takraata hai; sirf leaked fraction $m_{L 1}$ hi L2 tak pahunchta hai. "Local" us rung pe leakage measure karta hai relative to jo pahuncha; "global" leakage measure karta hai relative to top pe original full pour.

Topic ko yeh kyun chahiye: recursive formula $H T_{L 1} + m_{L 1} (H T_{L 2} + m_{L 2} M P_{L 2})$ andar local rate use karta hai, kyunki har cache sirf wahi accesses dekhta hai jo usse forward hue. Yeh distinction multi-level tuning ka sabse tricky hissa hai; local/global ulta karna ek classic bug hai.

Prerequisite map

Ise bottom-up padhो: ek access se hit ya miss milta hai; un se do times aur ek fraction milta hai; unka weighted average hi AMAT hai; phir subscripts aur local/global use levels mein nest karne dete hain.

Sab kuch anchor karne ke liye ek number

Picture (upar): parent ka Example 1 — $H T = 1$ ns, $m = 0.05$ , $M P = 100$ ns. Bar dikhata hai AMAT $= 6$ ns, jo 1 ns (jo tum hamesha pay karte ho, hit pe) aur 5 ns (jo rare misses contribute karte hain, $0.05 \times 100$ ) mein banta hai. Notice karo shocker: 6 mein se 5 ns misses se aate hain jo sirf 5% accesses hain. Yahi reason hai ki poora topic $m$ pe itna obsess karta hai.

Equipment checklist

Daayaan side cover karo aur khud test karo. Agar koi jawab fuzzy lage, toh parent note se pehle woh section phir padho.

"Ek memory access" kya hai?

CPU ki ek request single address pe data ke liye; woh cheez jo AMAT time karta hai.

Cache hit kya hai?

Data cache mein mil gaya — chhota, fast path.

Cache miss kya hai?

Data cache mein nahi tha, toh hum slow ladder ke aur neeche jaate hain.

H T

(hit time) kya measure karta hai?

Cache check karne aur hit pe data deliver karne ka time — har access pe pay hota hai.

M P

(miss penalty) kya measure karta hai?

Hit check ke baad block ko next level se fetch karne ka extra time.

Miss pe total time?

H T + M P

— tumne pehle hit check pay kiya, phir extra trip.

Miss rate

m

kya hai?

Un accesses ka fraction jo miss karte hain,

[0, 1]

mein ek number; misses ÷ total accesses.

Hit rate kya hai?

1 - m

, un accesses ka fraction jo hit karte hain.

Weighted average kya hai?

p \cdot a + q \cdot b

— har outcome ki cost ko us ke hone ki frequency se weight karke.

AMAT mein

H T

m

se multiply kyun nahi kiya?

Kyunki hits aur misses dono

H T

pay karte hain, isliye woh factor out ho jaata hai ek bare

+ H T

mein.

L2 ka local miss rate?

L2 misses ÷ woh accesses jo L2 tak pahunche.

L2 ka global miss rate?

L2 misses ÷ saare CPU accesses

= m_{L 1} \times m_{L 2, local}

Recursive AMAT ke andar kaunsa miss rate jaata hai?

Local rate — har level sirf wahi dekhta hai jo usse forward hua.

Connections

Average memory access time (AMAT) — woh parent note jiske liye yeh page tumhe ready karta hai.
Cache Miss Rate & Miss Types (3 Cs) — $m$ kahan se aata hai.
Miss Penalty & Main Memory Latency — $M P$ ko bada kya banata hai.
Multi-level Cache Hierarchy — ladder aur uske rungs.
Cache Associativity & Hit Time tradeoff — $H T$ vs $m$ tuning.
CPU Performance Equation — jahan AMAT overall speed mein feed hota hai.
Locality of Reference — hit rates itni high kyun hoti hain aakhir.