Exercises — Neuromorphic computing

(a) = co-located memory + compute (this is In-memory computing, the opposite of the von Neumann split). (b) = event-driven / sparse activity. (c) = massive parallelism. These are the three reasons the brain runs on ~20 W.

Data is encoded in the timing / rate of spikes (a temporal code), not as a dense float in a register. Activity is event-driven: a unit only does work when a spike arrives — no clock tick is spent on idle units. See Spiking Neural Networks (SNN).

The Leaky Integrate-and-Fire (LIF) neuron.

• Continuous part: the RC differential equation ${\tau }_m\dot{V}=-V+RI$ (integrate + leak).
• Discrete part: the fire-and-reset rule "if $V\ge V_{th}$, emit a spike and set $V\leftarrow V_{reset}$". The continuous part alone can never spike; the threshold makes it spiking.

What/Why: ${\tau }_m=RC$ sets how fast the membrane charges — nothing else can be computed without it. ${\tau }_m=(20\times 10^6)(0.5\times 10^{-9})=10\times 10^{-3}\text{s}=10\text{ms}.$

Why check $RI$ first? The voltage can only ever climb toward its steady value $RI$; if that ceiling is below threshold, it can never fire. $RI=(20\times 10^6)(1\times 10^{-9})=20\times 10^{-3}\text{V}=20\text{mV}.$ Since $20\text{mV}>12\text{mV}=V_{th}$ → it will fire. ✅

Why the log formula? We set $V(T)=V_{th}$ in the charging curve and solve for $T$. $T={\tau }_m\ln \frac{RI}{RI-V_{th}}=10\text{ms}\cdot \ln \frac{20}{20-12}=10\text{ms}\cdot \ln \mathrm{2.5.}$ $\ln 2.5\approx 0.9163$, so $T\approx 9.16\text{ms}$. $f=\frac{1}{T}\approx \frac{1}{9.16\times 10^{-3}\text{s}}\approx 109\text{Hz}.$

$RI=(20\times 10^6)(0.5\times 10^{-9})=10\text{mV}<12\text{mV}=V_{th}$. The log argument $\frac{10}{10-12}=\frac{10}{-2}$ is negative → never fires. It charges toward and rests at $RI===10\text{mV}==$ forever (the leak exactly balances the input there). Meaning: on weak input the neuron is silent, so it consumes ~zero dynamic energy. This is the sparsity that gives neuromorphic chips their energy edge.

As $I\to \infty$, $RI\gg V_{th}$, so $\frac{RI}{RI-V_{th}}\to 1$ and $\ln (\cdot )\to 0$. Then $T\to {\tau }_m\cdot 0=0$… but how it approaches matters: for large $RI$, $T\approx {\tau }_m\cdot \frac{V_{th}}{RI},$ using $\ln (1+x)\approx x$ with $x=\frac{V_{th}}{RI-V_{th}}$. So $f=1/T\approx \frac{RI}{{\tau }_m{V}_{th}}$ grows roughly linearly in $I$ at first, but a real neuron adds a fixed refractory period $t_{ref}$: $T_{total}=T+t_{ref}$. Once $T\ll t_{ref}$, the rate flattens to the ceiling $f_{max}=1/t_{ref}$. The curve below shows the knee.

Why $\Delta t=t_{post}-t_{pre}$ first? Its sign chooses the branch. (a) $\Delta t=10-2=+8\text{ms}>0$ → potentiation: $\Delta w=A_{+}{e}^{-\Delta t/{\tau }_{+}}=0.1e^{-8/20}=0.1e^{-0.4}\approx 0.1(0.6703)=+\mathrm{0.0670.}$ Pre came before post → it helped cause the spike → strengthen. (b) $\Delta t=2-10=-8\text{ms}<0$ → depression: $\Delta w=-A_{-}{e}^{+\Delta t/{\tau }_{-}}=-0.12e^{-8/20}=-0.12(0.6703)\approx -\mathrm{0.0804.}$ Pre came after post → it couldn't have caused the spike → weaken.

What/Why: we know $T$ and want $RI$, so invert $T={\tau }_m\ln \frac{RI}{RI-V_{th}}$. $\frac{T}{{\tau }_m}=\ln \frac{RI}{RI-V_{th}}\Rightarrow e^{T/{\tau }_m}=\frac{RI}{RI-V_{th}}.$ Let $k=e^{T/{\tau }_m}=e^{5/10}=e^{0.5}\approx 1.6487$. Then $k(RI-V_{th})=RI$, so $RI(k-1)=k{V}_{th}\Rightarrow RI=\frac{k}{k-1}{V}_{th}.$ $RI=\frac{1.6487}{0.6487}\times 15\text{mV}\approx 2.541\times 15\approx 38.1\text{mV}.$ Check: $\frac{RI}{RI-V_{th}}=\frac{38.1}{23.1}\approx 1.649=k$ ✓.

Neuromorphic (sparse): active neurons $=0.02\times 10^6=2\times 10^4$. Each fires $10$ spikes in $1\text{s}$, so total spikes $=2\times 10^4\times 10=2\times 10^5$. $E_{neuro}=2\times 10^5\times 20\text{pJ}=4\times 10^6\text{pJ}=4\mu \text{J}.$ Dense: $10^6$ neurons $\times$ $1000$ evaluations/s $=10^9$ ops. $E_{dense}=10^9\times 20\text{pJ}=2\times 10^{10}\text{pJ}=20,000\mu \text{J}=20\text{mJ}.$ Ratio: $E_{dense}/E_{neuro}=20\text{mJ}/4\mu \text{J}=5000\times$. Event-driven sparsity buys ~3.7 orders of magnitude here.

Step 1 — target period. $f=50\text{Hz}\Rightarrow T=20\text{ms}$. Step 2 — check the ceiling. $RI=(10^7)(2\times 10^{-9})=20\text{mV}>15\text{mV}$ ✓, so firing is possible. Step 3 — solve for ${\tau }_m$ from $T={\tau }_m\ln \frac{RI}{RI-V_{th}}$: $\ln \frac{20}{20-15}=\ln 4\approx 1.3863,{\tau }_m=\frac{T}{\ln 4}=\frac{20\text{ms}}{1.3863}\approx 14.43\text{ms}.$ Step 4 — get $C$ from ${\tau }_m=RC$: $C=\frac{{\tau }_m}{R}=\frac{14.43\times 10^{-3}}{10^7}\approx 1.443\times 10^{-9}\text{F}\approx 1.44\text{nF}.$ Step 5 — STDP with $\Delta t=T=20\text{ms}$ (pre before post): $\Delta w=A_{+}{e}^{-\Delta t/{\tau }_{+}}=0.1e^{-20/20}=0.1e^{-1}\approx 0.1(0.3679)=+\mathrm{0.0368.}$ So the synapse strengthens by about $0.037$ per such pairing. Because $\Delta t={\tau }_{+}$ exactly, the update is exactly $A_{+}/e$ — a clean sanity check.

• No bus: the synaptic weight lives at the connection (a memristor conductance, Memristors and ReRAM). Multiply-by-weight happens as current flows through it — In-memory computing. Data never travels to a separate ALU, so the von Neumann bottleneck vanishes. Cost: weights are analog and noisy; precision is limited.
• No global error: STDP updates each weight from only its own $\Delta t=t_{post}-t_{pre}$. There is no backward pass sweeping a loss across the whole network. Cost: it is local and unsupervised, so it learns correlations well but struggles with tasks needing precise global credit assignment (where GPUs + backprop still win — see GPU vs Neuromorphic accelerators).
• Sparsity: silent neurons ($RI\le V_{th}$) cost near-zero power. Cost: if a task needs dense, high-rate activity, you lose the advantage and pay per spike.