Neuromorphic computing: Exercises — Hinglish deep dive

6.5.14 · D4 · Hardware › Advanced & Emerging Architectures › Neuromorphic computing

Symbols se pehle, ek one-line dictionary taaki kuch bhi bina samjhe use na ho:

$V$ = membrane voltage (membrane capacitor par ikatta hua charge, volts mein).
$V_{t h}$ = threshold jis tak voltage ko spike emit karne ke liye pahunchna chahiye.
$R, C$ = leak resistance aur membrane capacitance (dekho RC circuits).
$τ_{m} = R C$ = time constant: kitne seconds mein membrane "mostly" charge ho jaati hai.
$I$ = neuron mein flow hone wala input current.
$Δ t$ = pre-to-post spike-time gap.

Har neuromorphic property ko us trick se match karo jo wo brain se copy karta hai: (a) synapse weight store bhi karta hai aur multiply bhi, (b) neurons fire hone tak chup rehte hain, (c) $1 0^{11}$ neurons ek saath run karte hain. Har trick ka naam batao.

Recall Solution

(a) = co-located memory + compute (yeh In-memory computing hai, von Neumann split ka ulta). (b) = event-driven / sparse activity. (c) = massive parallelism. Yahi teen wajah hain ki brain ~20 W par run karta hai.

L1·Q2

Niche di gayi table mein neuromorphic column bharo: data kaise encoded hota hai, aur activity kaise trigger hoti hai?

Recall Solution

Data spikes ki timing / rate mein encode hota hai (ek temporal code), na ki register mein ek dense float ki tarah. Activity event-driven hoti hai: ek unit tabhi kaam karta hai jab spike aati hai — idle units par koi clock tick spend nahi hota. Dekho Spiking Neural Networks (SNN).

L1·Q3

Parent note jo neuron model derive karta hai uska naam batao, aur uske do ingredients list karo: continuous part aur discrete part.

Recall Solution

Leaky Integrate-and-Fire (LIF) neuron.

Continuous part: RC differential equation $τ_{m} \dot{V} = - V + R I$ (integrate + leak).
Discrete part: fire-and-reset rule "agar $V \geq V_{t h}$ , spike emit karo aur $V \leftarrow V_{r ese t}$ set karo". Sirf continuous part kabhi spike nahi kar sakta; threshold hi ise spiking banata hai.

Level 2 — Application

L2·Q1

Ek LIF neuron hai jisme $R = 20 M Ω$ , $C = 0.5 nF$ . $τ_{m}$ nikalo.

Recall Solution

Kya/Kyun: $τ_{m} = R C$ batata hai ki membrane kitni fast charge hoti hai — bina iske aur kuch compute nahi ho sakta. $τ_{m} = (20 \times 1 0^{6}) (0.5 \times 1 0^{- 9}) = 10 \times 1 0^{- 3} s = 10 ms .$

L2·Q2

Wahi neuron, constant input $I = 1 nA$ , threshold $V_{t h} = 12 mV$ . Pehle, decide karo ki yeh fire karega ya nahi.

Recall Solution

Pehle $R I$ check kyun karein? Voltage sirf apni steady value $R I$ tak hi badh sakti hai; agar yeh ceiling threshold se neeche hai, toh yeh kabhi fire nahi kar sakta. $R I = (20 \times 1 0^{6}) (1 \times 1 0^{- 9}) = 20 \times 1 0^{- 3} V = 20 mV .$ Kyunki $20 mV > 12 mV = V_{t h}$ → yeh fire karega. ✅

L2·Q3

L2·Q2 aage badhao: pehli spike tak ka time $T$ aur firing rate $f = 1/ T$ nikalo.

Recall Solution

Log formula kyun? Hum charging curve mein $V (T) = V_{t h}$ set karte hain aur $T$ ke liye solve karte hain. $T = τ_{m} ln \frac{R I}{R I - V _{t h}} = 10 ms \cdot ln \frac{20}{20 - 12} = 10 ms \cdot ln 2.5.$ $ln 2.5 \approx 0.9163$ , toh $T \approx 9.16 ms$ . $f = \frac{1}{T} \approx \frac{1}{9.16 \times 1 0 ^{- 3} s} \approx 109 Hz .$

Charging curve dekho — neuron dashed ceiling $R I$ se kaafi pehle red threshold tak pahunch jaata hai:

Level 3 — Analysis

L3·Q1

L2 wala neuron lo ( $R = 20 M Ω$ , $τ_{m} = 10 ms$ , $V_{t h} = 12 mV$ ) lekin input $I = 0.5 nA$ kar do. Kya yeh fire karega? Agar nahi, toh yeh kis steady voltage par settle karega, aur iska silence power ke liye kya matlab rakhta hai?

Recall Solution

$R I = (20 \times 1 0^{6}) (0.5 \times 1 0^{- 9}) = 10 mV < 12 mV = V_{t h}$ . Log argument $\frac{10}{10 - 12} = \frac{10}{- 2}$ negative hai → kabhi fire nahi karega. Yeh $R I === 10 mV ==$ ki taraf charge hoga aur hamesha wahan ruka rahega (leak exactly wahan input ko balance karta hai). Matlab: weak input par neuron silent rehta hai, isliye dynamic energy ~zero consume karta hai. Yahi sparsity hai jo neuromorphic chips ko unka energy edge deti hai.

L3·Q2

Ek firing neuron ke liye firing rate $f = 1/ T$ input $I$ ke saath badhti hai. $T (I)$ ki shape use karte hue explain karo ki $f$ saturate kyun hoti hai (utni tezi se badhna band ho jaati hai) jab $I \to \infty$ .

Recall Solution

Jab $I \to \infty$ , $R I ≫ V_{t h}$ , toh $\frac{R I}{R I - V _{t h}} \to 1$ aur $ln (\cdot) \to 0$ . Phir $T \to τ_{m} \cdot 0 = 0$ … lekin kaise approach karta hai yeh matter karta hai: bade $R I$ ke liye, $T \approx τ_{m} \cdot \frac{V _{t h}}{R I},$ $ln (1 + x) \approx x$ use karke jahan $x = \frac{V _{t h}}{R I - V _{t h}}$ . Toh $f = 1/ T \approx \frac{R I}{τ _{m} V _{t h}}$ pehle $I$ mein roughly linearly badhti hai, lekin ek real neuron ek fixed refractory period $t_{r e f}$ add karta hai: $T_{t o t a l} = T + t_{r e f}$ . Jab $T ≪ t_{r e f}$ ho jaata hai, rate ceiling $f_{ma x} = 1/ t_{r e f}$ par flat ho jaati hai. Neeche wala curve knee dikhata hai.

L3·Q3

STDP with $A_{+} = 0.1$ , $τ_{+} = 20 ms$ , $A_{-} = 0.12$ , $τ_{-} = 20 ms$ . $Δ w$ compute karo: (a) pre at $2 ms$ , post at $10 ms$ ; (b) pre at $10 ms$ , post at $2 ms$ . Har sign ko interpret karo.

Recall Solution

Pehle $Δ t = t_{p os t} - t_{p r e}$ kyun? Iska sign branch choose karta hai. (a) $Δ t = 10 - 2 = + 8 ms > 0$ → potentiation: $Δ w = A_{+} e^{- Δ t / τ_{+}} = 0.1 e^{- 8/20} = 0.1 e^{- 0.4} \approx 0.1 (0.6703) = + 0.0670.$ Pre pehle aaya post se → usne spike cause karne mein help ki → strengthen. (b) $Δ t = 2 - 10 = - 8 ms < 0$ → depression: $Δ w = - A_{-} e^{+ Δ t / τ_{-}} = - 0.12 e^{- 8/20} = - 0.12 (0.6703) \approx - 0.0804.$ Pre baad mein aaya post ke → spike cause nahi kar sakta tha → weaken.

Level 4 — Synthesis

L4·Q1

Ek vision chip ko ek neuron exactly $T = 5 ms$ par fire karna hai jab constant input on hota hai. Aapko $τ_{m} = 10 ms$ aur $V_{t h} = 15 mV$ diya gaya hai. Input ko kaunsi steady value $R I$ produce karni chahiye? (Time-to-spike formula ko invert karo.)

Recall Solution

Kya/Kyun: hum $T$ jaante hain aur $R I$ chahiye, toh $T = τ_{m} ln \frac{R I}{R I - V _{t h}}$ invert karo. $\frac{T}{τ _{m}} = ln \frac{R I}{R I - V _{t h}} \Rightarrow e^{T / τ_{m}} = \frac{R I}{R I - V _{t h}} .$ Maan lo $k = e^{T / τ_{m}} = e^{5/10} = e^{0.5} \approx 1.6487$ . Phir $k (R I - V_{t h}) = R I$ , toh $R I (k - 1) = k V_{t h} \Rightarrow R I = \frac{k}{k - 1} V_{t h} .$ $R I = \frac{1.6487}{0.6487} \times 15 mV \approx 2.541 \times 15 \approx 38.1 mV .$ Check: $\frac{R I}{R I - V _{t h}} = \frac{38.1}{23.1} \approx 1.649 = k$ ✓.

L4·Q2

Energy reasoning ko sparsity ke saath combine karo. Ek neuromorphic layer mein $1 0^{6}$ neurons $1 s$ ke liye run kar rahe hain. Maan lo har spike ki cost $E_{s p} = 20 pJ$ hai, aur average mein sirf 2% neurons active hain, har ek $10 Hz$ par fire kar raha hai. Total spike energy estimate karo. Ek hypothetical dense chip se compare karo jo har $ms$ par sab $1 0^{6}$ neurons ko $20 pJ$ each par "evaluate" karta hai.

Recall Solution

Neuromorphic (sparse): active neurons $= 0.02 \times 1 0^{6} = 2 \times 1 0^{4}$ . Har ek $1 s$ mein $10$ spikes fire karta hai, toh total spikes $= 2 \times 1 0^{4} \times 10 = 2 \times 1 0^{5}$ . $E_{n e u r o} = 2 \times 1 0^{5} \times 20 pJ = 4 \times 1 0^{6} pJ = 4 μ J .$ Dense: $1 0^{6}$ neurons $\times$ $1000$ evaluations/s $= 1 0^{9}$ ops. $E_{d e n se} = 1 0^{9} \times 20 pJ = 2 \times 1 0^{10} pJ = 20, 000 μ J = 20 mJ .$ Ratio: $E_{d e n se} / E_{n e u r o} = 20 mJ /4 μ J = 5000 \times$ . Event-driven sparsity yahan ~3.7 orders of magnitude faayda deta hai.

Level 5 — Mastery

L5·Q1

Design + predict. Aapke paas ek RC-based LIF neuron hai aur aap constant input $I = 2 nA$ ke under ~ $50 Hz$ firing rate chahte ho, $V_{t h} = 15 mV$ aur $R = 10 M Ω$ use karke. $C$ (aur isse $τ_{m}$ ) choose karo. Phir, STDP with $A_{+} = 0.1$ , $τ_{+} = 20 ms$ use karke, ek pre-post gap ke liye potentiation predict karo jo ek firing period $T = 1/ f$ ke equal ho.

Recall Solution

Step 1 — target period. $f = 50 Hz \Rightarrow T = 20 ms$ . Step 2 — ceiling check. $R I = (1 0^{7}) (2 \times 1 0^{- 9}) = 20 mV > 15 mV$ ✓, toh firing possible hai. Step 3 — $τ_{m}$ solve karo $T = τ_{m} ln \frac{R I}{R I - V _{t h}}$ se: $ln \frac{20}{20 - 15} = ln 4 \approx 1.3863, τ_{m} = \frac{T}{l n 4} = \frac{20 ms}{1.3863} \approx 14.43 ms .$ Step 4 — $C$ nikalo $τ_{m} = R C$ se: $C = \frac{τ _{m}}{R} = \frac{14.43 \times 1 0 ^{- 3}}{1 0 ^{7}} \approx 1.443 \times 1 0^{- 9} F \approx 1.44 nF .$ Step 5 — STDP with $Δ t = T = 20 ms$ (pre before post): $Δ w = A_{+} e^{- Δ t / τ_{+}} = 0.1 e^{- 20/20} = 0.1 e^{- 1} \approx 0.1 (0.3679) = + 0.0368.$ Toh synapse har aisi pairing mein ~ $0.037$ se strengthen hota hai. Kyunki $Δ t = τ_{+}$ exactly hai, update exactly $A_{+} / e$ hai — ek clean sanity check.

L5·Q2

Sab kuch ek saath jodke explain karo, kyun is LIF+STDP system ko koi bus aur koi global error signal nahi chahiye — aur har choice ki kya cost hai.

Recall Solution

No bus: synaptic weight connection par hi rehta hai (ek memristor conductance, Memristors and ReRAM). Multiply-by-weight tab hota hai jab current usme se flow hoti hai — In-memory computing. Data kabhi alag ALU tak travel nahi karta, isliye von Neumann bottleneck khatam ho jaata hai. Cost: weights analog aur noisy hote hain; precision limited hoti hai.
No global error: STDP har weight ko sirf apne $Δ t = t_{p os t} - t_{p r e}$ se update karta hai. Poore network mein loss sweep karne wala koi backward pass nahi hota. Cost: yeh local aur unsupervised hai, isliye correlations achhe se seekhta hai lekin un tasks mein struggle karta hai jinhein precise global credit assignment chahiye (jahan GPUs + backprop abhi bhi jeetat hain — dekho GPU vs Neuromorphic accelerators).
Sparsity: silent neurons ( $R I \leq V_{t h}$ ) near-zero power cost karte hain. Cost: agar kisi task ko dense, high-rate activity chahiye, toh advantage kho dete ho aur per spike pay karte ho.

Exercises — Neuromorphic computing

Level 1 — Recognition

L1·Q1

L1·Q2

L1·Q3

Level 2 — Application

L2·Q1

L2·Q2

L2·Q3

Level 3 — Analysis

L3·Q1

L3·Q2

L3·Q3

Level 4 — Synthesis

L4·Q1

L4·Q2

Level 5 — Mastery

L5·Q1

L5·Q2