Visual walkthrough — Neuromorphic computing

6.5.14 · D2 · Hardware › Advanced & Emerging Architectures › Neuromorphic computing

---

Step 1 — Ek membrane ek saath do cheezein hai: ek bucket aur ek leak

KYA. Ek neuron ki thin fatty wall ki picture lo. Ek taraf, positive charge pile up hoti hai; wall dono tarafon ko alag rakhti hai. Yahi kaam ek capacitor karta hai — woh store karta hai separated charge ko. Lekin wall perfect nahi hai: choti choti channels charge ko wapas trickle karne deti hain. Jo cheez charge ko trickle karne deti hai proportion mein ki kitna hard you push, woh hota hai resistor. Toh membrane ek resistor aur ek capacitor hai, side by side — ek RC circuit.

YEH picture kyun, plain wire kyun nahi. Ek plain wire ki koi memory nahi hoti: current push karo, charge flow karta hai, kuch accumulate nahi hota. Lekin ek neuron yaad rakhta hai recent input ko thodi der ke liye aur phir bhool jaata hai. Sirf ek store-and-leak object hi yeh kar sakta hai. RC circuit sabse simple object hai jisme dono hain — ek memory (capacitor) aur ek fade (resistor).

PICTURE. Left wala tank input current $I$ se bharta hai; right mein chhoti si hole se paani bahar leak hota hai. Paani ka level voltage $V$ hai.

---

Step 2 — Balance likhte hain: jo aata hai woh kahin jaata hai

KYA. Jo bhi current aati hai uske sirf do possible fates hain: ya toh woh capacitor par pile hoti hai (level raise karti hai) ya phir leak se escape karti hai. Kuch bhi vanish nahi hota. Yahi ek sentence poora equation hai: $I(t)=\underset{\text{fills the tank}}{\underbrace{C\frac{dV}{dt}}}+\underset{\text{leaks out}}{\underbrace{\frac{V}{R}}}$

YEH derivative yahan kyun aata hai. Tool $\frac{dV}{dt}$ bilkul naya hai, toh hum isko justify karte hain. Iska matlab hai "level $V$ abhi kitni tezi se change ho rahi hai." Hume yeh chahiye kyunki ek capacitor ko charge karne wali current iske baare mein nahi hai ki woh kitna bhara hai — woh iske baare mein hai ki woh kitni tezi se bhar raha hai. Sawaal "yeh kitni tezi se change ho raha hai?" exactly woh sawaal hai jiska jawab derivative deta hai, toh physics hume yahi tool use karne par majboor karti hai.

Term by term:

• $C\frac{dV}{dt}$ — charge jo storage mein ja rahi hai. Wider tank ($C$ bada) ya faster rise ($\frac{dV}{dt}$ bada) ka matlab zyada current swallow ho rahi hai.
• $\frac{V}{R}$ — leak. Zyada level $V$ drain se harder push karta hai; narrow hole (bada $R$) kam leak karta hai.

PICTURE. Incoming arrow do mein split hoti hai: ek tank mein, ek leak se bahar. Unke sizes ka sum input ke barabar hona chahiye.

---

Step 3 — Clean up karo, aur "forgetting time" ${\tau }_m$ se milo

KYA. Har term ko $R$ se multiply karo aur gather karo: ${\tau }_m\frac{dV}{dt}=-V+RI(t),{\tau }_m\equiv RC$

${\tau }_m=RC$ kyun define karein. Hum $R$ aur $C$ ko ek symbol mein bundle karte hain kyunki saath milkar yeh system mein ek hi timescale set karte hain. $R$ (ohms) times $C$ (farads) seconds mein aata hai — yeh literally ek duration hai. Hum ise ${\tau }_m$ kehte hain, membrane time constant: neuron ka "attention span."

$V$ ke saamne minus sign kyun. Input band karo ($I=0$). Equation ban jaati hai ${\tau }_m\dot{V}=-V$: jitna zyada bhara tank, utni tezi se khaali hoga. Woh minus sign hi leak hai — yeh hamesha $V$ ko zero ki taraf drag karta hai. Yahi "leaky" hai Leaky Integrate-and-Fire mein: purana input continuously bhula diya jaata hai.

PICTURE. Tap band karne par, level ek smooth curve pe neeche slide karta hai; ek ${\tau }_m$ ke baad woh apne starting point ka lagbhag $37\%$ rah jaata hai (woh $0.37$ hai $1/e$, ek number jise hum properly agle step mein milenge).

Cloze check:

---

Step 4 — Steady tap chalao: charging curve

KYA. Input ko constant $I$ par rakho, empty start karo ($V(0)=0$). ${\tau }_m\dot{V}=-V+RI$ solve karne par milta hai: $V(t)=RI\left(1-e^{-t/{\tau }_m}\right)$

YEH number $e$ kyun aata hai. Humne exponential choose nahi ki — leak ne ki. Jab bhi kisi cheez ki rate of change khud uske proportional hoti hai ("jitna zyada hai, utna tezi se move karta hai"), sirf ek hi function usse satisfy karta hai: $e^{(\dots )}$. Yahi $e$ ka job description hai: woh function hai jo apna khud ka rate of change hai. Leak $\dot{V}$ ko $V$ par depend karvata hai, toh $e$ unavoidable hai.

Term by term:

• $RI$ — ceiling. Jab $t\to \infty$, bracket $\to 1$, toh $V\to RI$. Yeh woh level hai jahan "in" exactly "leak" ko balance karta hai, toh filling ruk jaati hai. Yeh value yaad rakho; Step 6 isi par hang karta hai.
• $e^{-t/{\tau }_m}$ — approach. $t=0$ par yeh $1$ hai (bracket $=0$, empty). Waqt ke saath shrink hota hai, toh bracket $1$ ki taraf climb karta hai.
• $t={\tau }_m$ ke baad: $e^{-1}\approx 0.37$, toh $V\approx 0.63RI$ — ek time constant mein 63% upar.

PICTURE. Ek rising curve jo steep shuru hoti hai aur dashed ceiling $RI$ ki taraf flatten hoti hai. ${\tau }_m$ par tick 63% point mark karta hai.

---

Step 5 — Trigger add karo: threshold, fire, reset

KYA. Step 4 ki smooth curve kabhi actually spike nahi karti — woh bas $RI$ tak glide karti hai. Ek neuron ko ek sharp event produce karna chahiye. Toh hum ek rule haath se bolt on karte hain: $\text{if }V(t)\ge V_{th}\Rightarrow \text{emit a spike, then set }V\leftarrow V_{reset}$

YEH ODE se alag kyun hai. Differential equation analog aur smooth hai; spike digital aur abrupt hai. Koi bhi smooth equation all-or-nothing event nahi bana sakti, toh threshold ek extra ingredient hai. Yeh bolt-on precisely wahi hai jo ek RC filter ko ek spiking neuron mein convert karta hai — kisi bhi SNN ka dil.

PICTURE. Charging curve climb karti hai, coral line $V_{th}$ ko kiss karti hai, ek vertical spike upar shoot karta hai, aur $V$ wapas $V_{reset}$ par drop ho jaata hai dobara charge karne ke liye.

---

Step 6 — Time-to-spike ke liye solve karo (aur logarithm se milo)

KYA. Neuron ठीक उसी instant fire karta hai jab curve $V_{th}$ tak pahunchti hai. Us instant ko $T$ kahte hain. Step-4 curve mein $V(T)=V_{th}$ set karo aur unravel karo: $V_{th}=RI\left(1-e^{-T/{\tau }_m}\right)\Rightarrow e^{-T/{\tau }_m}=1-\frac{V_{th}}{RI}$ $\boxed{T={\tau }_m\ln \frac{RI}{RI-V_{th}}}$

AB logarithm kyun. Hamare paas ek equation hai jahan unknown $T$ ek exponent ke andar phansa hua hai: $e^{-T/{\tau }_m}$. Use free karne ke liye hume woh tool chahiye jo ek exponential ko undo kare — exactly yahi $\ln$ (natural logarithm) ka kaam hai: yeh answer deta hai "is number ko paane ke liye $e$ ko kis power par raise karein?" Dono sides par $\ln$ apply karne se $T$ exponent se neeche aa jaata hai. Sirf yahi ek tool hai jo yeh kaam cleanly karta hai, isliye yeh yahan appear hota hai.

Box ke andar term by term:

• Saamne ${\tau }_m$ — sab kuch scale karta hai: ek sluggish membrane (bada ${\tau }_m$) fire karne mein proportionally zyada time leta hai.
• $\frac{RI}{RI-V_{th}}$ — "ceiling" aur "remaining gap" ka ratio. Jitna close ceiling $RI$ threshold $V_{th}$ ke paas hogi, utna chota denominator, utna bada log, utna zyada time lagega.

PICTURE. Wahi curve, ab $T$ mark kiya hua jahan woh $V_{th}$ cross karti hai, aur ek bracket shrinking gap $RI-V_{th}$ dikha raha hai.

---

Step 7 — Degenerate case: jab neuron KABHI fire nahi karta

KYA. Denominator $RI-V_{th}$ dekho. Kya ho agar ceiling $RI$ threshold se neeche ho?

• Agar $RI<V_{th}$: fraction $\frac{RI}{RI-V_{th}}$ ka negative denominator hai → poora argument negative → negative number ka $\ln$ exist nahi karta. Mathematically formula break ho jaata hai; physically, curve coral line ke neeche flatten ho jaati hai aur kabhi touch nahi karti. Koi spike kabhi nahi.
• Agar $RI=V_{th}$: denominator $0$ hai, fraction blow up ho jaata hai, $\ln (\infty )=\infty$ → $T\to \infty$. Neuron sirf just infinite time ke baad threshold tak pahunchta hai — practically silent hi hai.

YEH kyun matter karta hai — yahi poori energy story hai. Ek neuron jiska steady level threshold ke neeche ho practically koi power nahi jalata: woh koi spikes produce nahi karta, toh koi downstream circuits wake up nahi hote. Yahi woh sparsity hai jo brains (aur neuromorphic chips) ko ek light bulb ki jitni power par run karne deti hai, unlike ek von Neumann machine jo har cell clock karta hai chahe kaam ho ya na ho.

PICTURE. Do curves side by side: ek ceiling $V_{th}$ ke upar (fires), ek neeche (flat, hamesha ke liye silent).

---

Step 8 — Ek number daalo: kya yeh neuron fire karta hai, aur kitni tezi se?

KYA. Lo $R=10\text{M}\Omega$, $C=1\text{nF}$, $V_{th}=15\text{mV}$, $I=2\text{nA}$.

YEH steps is order mein kyun karein. Pehle ceiling (kya yeh fire bhi karta hai?), phir timescale, phir woh time.

1. Ceiling $RI=(10\times 10^6)(2\times 10^{-9})=20\text{mV}$. Kyunki $20>15=V_{th}$ → yeh fire karta hai. ✅ (Step 7 pass.)
2. Timescale ${\tau }_m=RC=(10^7)(10^{-9})=0.01\text{s}=10\text{ms}$.
3. Time-to-spike $T={\tau }_m\ln \frac{RI}{RI-V_{th}}=10\text{ms}\cdot \ln \frac{20}{20-15}=10\text{ms}\cdot \ln 4\approx 13.9\text{ms}$.
4. Rate $f=1/T\approx 72\text{Hz}$.

Agar hum $I=1\text{nA}$ par drop karein: $RI=10\text{mV}<15\text{mV}$ → argument negative → kabhi fire nahi karta (Step 7 ka silent branch).

PICTURE. In numbers ke liye actual charging curve, $15\text{mV}$ cross karte hue lagbhag $13.9\text{ms}$ par, saath mein $1\text{nA}$ curve jo threshold ke neeche flat rehti hai.

---

Ek-picture summary

Is page ka har idea ek single frame mein: current in → tank ek $e$-curve ke along ceiling $RI$ ki taraf charge hota hai → $V_{th}$ cross karta hai time $T={\tau }_m\ln \frac{RI}{RI-V_{th}}$ par → spike → reset → repeat; aur agar $RI<V_{th}$, curve line ke neeche flatten ho jaati hai aur hamesha ke liye silent rehti hai.

Recall Feynman retelling — ise ek story ki tarah kaho

Ek neuron ek leaky bucket hai. Current ek tap hai jo ise bharta hai; side mein ek hole se paani dribble karta hai. Paani ka level voltage hai. Hole ki wajah se level hamesha ke liye nahi barhta — pehle tezi se climb karta hai, phir ek ceiling tak ease up ho jaata hai jahan dripping-in exactly dripping-out ko match karta hai. Woh ceiling $RI$ hai. Bucket ki ek memory span hai, ${\tau }_m=RC$: tap band karo aur yeh kuch ${\tau }_m$ mein bhool jaata hai. Ab height $V_{th}$ par ek red line paint karo. Agar ceiling line ke upar hai, paani ek aise moment par use cross karta hai jo hum compute kar sakte hain — woh crossing ek spike hai, aur baad mein hum bucket dump karte hain aur dobara shuru karte hain. Crossing time mein ek logarithm use hota hai, kyunki hume time ko ek exponent ke andar se un-trap karna tha. Agar ceiling red line ke neeche baith jaati hai, paani kabhi use touch nahi karta: koi spike nahi, koi energy nahi lagti. Woh silence — ek bucket jo quietly still rehta hai — woh trick hai jo in buckets ke pore ek ocean ko almost bina power ke run karne deti hai. Dekho In-memory computing aur Hebbian learning ke liye ki kya hota hai jab buckets buckets se baat karne lagte hain.

Recall Quick self-test

$V(t)$ mein exponential (straight line nahi) kyun aata hai? ::: Kyunki leak $V$ ke rate of change ko $V$ ke proportional bana deta hai, aur $e$ woh unique function hai jiska rate of change khud uske proportional hai. $T$ ke liye solve karte waqt logarithm kyun aata hai? ::: Unknown $T$ ek exponent ke andar phansa hua hai; $\ln$ woh tool hai jo $e^{(\cdot )}$ ko undo karta hai aur use free karta hai. Neuron kabhi fire kyun nahi karta? ::: Jab $RI\le V_{th}$: ceiling threshold par/neeche hoti hai, toh curve kabhi use cross nahi karti — $\ln$ argument non-positive ho jaata hai. ${\tau }_m$ physically kya hai? ::: $RC$, membrane ka forgetting time; ek ${\tau }_m$ ke baad voltage apni ceiling ki taraf 63% move ho chuki hoti hai.

---

See also: GPU vs Neuromorphic accelerators, Memristors and ReRAM (kaise ek synapse physically apna weight compute ke paas store karta hai).