Ek single database server ki hard limits hoti hain: finite disk, finite RAM, finite CPU, aur woh crash bhi kar sakta hai. Isse do alag problems nikalti hain:
Size/throughput ki problem → koi single box 50 TB store nahi kar sakta ya 1M writes/sec serve nahi kar sakta. → Sharding (a.k.a. horizontal partitioning): dataset ko disjoint pieces mein kaat do.
Failure/availability ki problem → agar woh ek box mar jaaye, tumhara app mar jaata hai. → Replication: doosre boxes pe redundant copies rakho.
Kyun acha hai: ek achha hash keys ko uniformly spread karta hai → koi hotspots nahi.
Kyun bura hai: range queries saare shards pe scatter ho jaati hain; aur N badalne se almost sab kuch remap ho jaata hai.
HOW resharding ka dard aata hai (derive karo):shard = hash(k) mod N ke saath, ek key k change N→N+1 ke paas tabhi tiki rehti hai jab
hash(k)modN=hash(k)mod(N+1).
"Random" hash values ke liye, ek key apna slot sirf lagbhag N+11 probability se rakhti hai (usse usi ek residue class mein N+1 mein se waapis aana padta hai). Toh keys ka jo fraction move karna padta hai woh hai
fmoved≈1−N+11=N+1N(lagbhag saari keys move karti hain).
Yeh catastrophic hai — har shard reshuffle ho jaata hai. Yahi problem consistent hashing solve karta hai.
Synchronous: leader follower ka confirm aane tak wait karta hai client ko ack karne se pehle → failover pe koi data loss nahi, lekin slower hai aur agar follower down hai toh stall ho jaata hai.
Asynchronous: leader turant ack karta hai → fast, lekin crash last un-replicated writes lose kar sakta hai (replication lag).
Sharding kaun si problem solve karta hai vs replication?
Sharding scale solve karta hai (data/throughput ek node ke liye bahut bada, distinct data split karo); replication availability/durability solve karta hai (usi data ki copies failures survive karti hain).
Shard key define karo.
Woh attribute jiska value partition function mein daala jaata hai yeh decide karne ke liye ki kaun sa shard kisi given row ki ownership rakhta hai.
Range sharding hotspots kyun create kar sakta hai?
Sequential/time-ordered keys saari last range mein fall karti hain, toh saari nayi writes ek shard pe hit karti hain.
N se N+1 shards jaane pe naive hash mod N ke under kitni fraction keys move hoti hain?
Lagbhag N/(N+1) — lagbhag saari keys.
Consistent hashing mein node add karne pe kitni fraction keys move hoti hain?
Lagbhag 1/(N+1) — sirf naye node ka arc.
Consistent hashing ek key ko node se kaise assign karta hai?
Nodes aur keys ko hash ring pe rakho; key ki ownership us pehle node ke paas hoti hai jo key se clockwise ho.
Virtual nodes kis liye hote hain?
Ek physical node ke bahut saare ring positions taaki load even ho aur rebalancing smooth ho.
Quorum consistency condition batao.
W + R > N guarantee karta hai ki read set aur write set overlap karega, toh ek read latest write dekhega.
W+R>N kyun kaam karta hai, derive karo.
Ek N-set ke do subsets ≥ W+R−N se overlap karte hain; ≥1 overlap chahiye toh W+R>N milta hai (pigeonhole).
Synchronous vs asynchronous replication ka trade-off?
Sync = failover pe koi data loss nahi lekin slower / slow followers pe block; async = fast lekin recent writes lose ho sakti hain (replication lag).
Single-leader vs multi-leader ka main difference?
Single-leader mein ek writer hota hai (koi conflicts nahi, lekin bottleneck/SPOF); multi-leader mein kai jagah writes allow hote hain (low-latency, lekin write conflicts).
N=3 ke saath, W=1,R=1 unsafe kyun hai?
1+1=2 jo > 3 nahi hai, toh read aur write sets ek doosre ko miss kar sakte hain → stale reads.
Network partition ke dauran CAP kya force karta hai?
Consistency choose karo (possibly-stale answers se mana karo) ya Availability (phir bhi answer do), dono nahi.
Recall Feynman: ek 12-saal ke bacche ko samjhao
Socho tumhara toy collection ek box mein nahi aati. Sharding matlab toys ko kai boxes mein split karna — cars box 1 mein, dolls box 2 mein — har toy exactly ek box mein rehti hai, toh koi bhi ek box light rehta hai. Lekin boxes toot sakte hain! Toh replication matlab har box ka content photocopy karna aur doosre rooms mein rakhna — ek toota toh copy survive karta hai. Ab toy jaldi dhundhne ke liye hum ek magic rule use karte hain: ek ring with arrows, aur har toy next box clockwise ki hoti hai. Cool part: ek naya box add karo aur sirf toys ka ek chhota slice move karna padta hai, sab kuch nahi. Aur jab friends poochhe "kya yeh newest toy hai?", hum ensure karte hain ki jis box mein save kiya aur jis box se padha woh hamesha kam se kam ek box share karein — yahi rule hai W+R>N.
