Noesis
6.5.14Advanced & Emerging Architectures

Neuromorphic computing

2,429 words11 min readdifficulty · medium
🇮🇳 Yeh note Hinglish mein padho →

WHY does neuromorphic computing exist?

Neuromorphic hardware copies these three tricks in silicon.

WHAT is it, precisely?

Key contrasts

Feature von Neumann (CPU/GPU) Neuromorphic
Memory & compute Separate, bus between Co-located
Activity Clocked, always on Event-driven (sparse)
Data Dense floats Sparse spikes
Encoding Value in a register Timing / rate of spikes
Energy High (data movement) Very low

HOW does a neuron compute? — Derive the LIF model from scratch

The workhorse is the Leaky Integrate-and-Fire (LIF) neuron. Let's build it from physics, not memorize it.

Figure — Neuromorphic computing

HOW do synapses learn? — STDP from first principles

Worked examples

Common mistakes (Steel-manned)

The chips (the 20% worth knowing)

  • IBM TrueNorth — 1M neurons, 256M synapses, ~70 mW; fully digital, event-driven.
  • Intel Loihi / Loihi 2 — on-chip learning (programmable STDP), asynchronous cores.
  • SpiNNaker (Manchester) — many ARM cores simulating spikes with AER messaging.
  • Memristor crossbars — do the multiply-accumulate in analog, in-memory (Ohm's law does the multiply, Kirchhoff's law does the sum). This is the physical embodiment of co-located memory+compute.

Flashcards

What bottleneck does neuromorphic computing attack?
The von Neumann bottleneck — energy/time wasted moving data between separate memory and compute.
Name the three brain tricks neuromorphic hardware copies.
Co-located memory+compute, event-driven sparsity, massive parallelism.
Roughly how many neurons and synapses in the human brain?
~101110^{11} neurons and ~101410^{14} synapses (about 100–150 trillion).
Write the LIF membrane equation.
τm dV/dt=−V+RI(t)\tau_m\,dV/dt = -V + RI(t), with τm=RC\tau_m = RC.
Why is the LIF neuron called "leaky"?
With no input, V˙=−V/τm\dot V = -V/\tau_m so VV decays exponentially — the neuron forgets past input.
Formula for time-to-first-spike under constant current.
T=τmln⁡RIRI−VthT = \tau_m \ln\frac{RI}{RI - V_{th}}.
When does a LIF neuron never fire?
When steady-state voltage RI≤VthRI \le V_{th} (log argument non-positive).
What is STDP?
Spike-Timing-Dependent Plasticity: synapse strengthens if pre fires before post, weakens if after; magnitude decays exponentially with ∣Δt∣|\Delta t|.
Sign convention: Δt=tpost−tpre>0\Delta t = t_{post}-t_{pre} > 0 means?
Pre before post → potentiation (weight increases).
What does AER stand for and do?
Address-Event Representation: encodes which neuron spiked and when, so only active events are transmitted. It is a common but not universal neuromorphic protocol.
What is encoded in a spike train (vs an ML neuron)?
Information in timing/rate of discrete spikes over time, not a single continuous value.
How do memristor crossbars compute a MAC in-memory?
Ohm's law gives current = conductance×voltage (multiply); Kirchhoff sums currents on a wire (accumulate).
Name two neuromorphic chips and a distinguishing feature.
IBM TrueNorth (~70 mW, digital, event-driven); Intel Loihi (on-chip programmable STDP learning).
Recall Feynman: explain to a 12-year-old

Imagine a class where kids only raise their hand ("spike") when they really have something to say, and they stay quiet otherwise — that saves everyone's energy. Each kid slowly "fills up with excitement" as friends whisper to them, and once full enough, they shout, then calm down again. Friends who whispered right before the shout become closer friends (their whisper mattered more). A normal computer instead makes every kid shout on every clock tick and carries all answers across the room to a teacher — exhausting. Neuromorphic chips are the quiet, only-when-needed classroom, and that's why they sip power like the brain.

Connections

  • Von Neumann architecture — the bottleneck neuromorphic escapes.
  • Memristors and ReRAM — analog in-memory synapses.
  • Spiking Neural Networks (SNN) — the computational model run on the hardware.
  • GPU vs Neuromorphic accelerators — dense vs event-driven compute.
  • RC circuits — the physics behind the LIF membrane.
  • Hebbian learning — biological root of STDP.
  • In-memory computing — the broader trend of co-locating storage and compute.

Concept Map

data movement wastes energy

inspires

motivates

co-located memory + compute

event-driven sparse

massive parallelism

implements

built from

stores locally

communicates via

modeled as

derived from

fires when V crosses Vth

von Neumann bottleneck

Efficiency problem

Brain ~20 watts

Neuromorphic computing

Spiking Neural Networks

Spiking neuron

Synaptic weights

Address-Event Representation

Leaky Integrate-and-Fire

RC circuit

Spike + reset

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho, normal computer mein memory aur processor alag-alag boxes hote hain, aur data baar-baar bus ke through idhar-udhar move karta rehta hai. Yahi hai von Neumann bottleneck — sabse zyada energy data move karne mein waste hoti hai, compute mein nahi. Ab brain dekho: sirf ~20 watt mein chalta hai, kyunki wahan memory aur compute ek hi jagah (synapse) par hain, aur neurons tabhi "spike" (electrical signal) bhejte hain jab zaroorat ho — warna chup rehte hain. Brain mein roughly 101110^{11} neurons aur 101410^{14} (lagbhag 100–150 trillion) synapses hote hain. Neuromorphic computing isi brain ke tarike ko silicon mein copy karta hai.

Core neuron model hai Leaky Integrate-and-Fire (LIF). Socho ek leaky balti: current (paani) andar aata hai, balti thodi-thodi leak bhi karti hai (−V-V wala term), aur jab paani threshold VthV_{th} tak pahunch jaye to neuron ek spike maarta hai aur reset ho jaata hai. Physics simple hai — membrane ek RC circuit hai, isliye equation banti hai τm dV/dt=−V+RI\tau_m\,dV/dt = -V + RI, jahan τm=RC\tau_m = RC. Important baat: agar steady-state voltage RIRI hi VthV_{th} se kam hai, to neuron kabhi fire hi nahi karega — yahi silence energy bachati hai.

Learning ke liye STDP hota hai: agar pre-neuron post-neuron se thoda pehle fire kare (matlab usne madad ki), to connection strong hota hai; agar baad mein kare to weak. Aur ye rule purely local hai — sirf apne do spike times chahiye, koi global backpropagation nahi. Isiliye hardware mein banana sasta aur efficient hai (jaise Intel Loihi, IBM TrueNorth chips).

Matlab yaad rakho: neuromorphic ka fayda dense matrix speed nahi, balki sparse, time-based data par bahut kam energy hai. Ye GPU ka replacement nahi, alag philosophy hai — brain jaisi, event-driven, memory-plus-compute-ek-jagah. Communication ke liye AER (Address-Event Representation) ek common protocol hai, lekin sabhi systems ise use nahi karte — ye mandatory nahi hai.

Go deeper — visual, from zero

D1FoundationsEvery symbol from zero · 3,058 wordsD2Visual walkthroughThe derivation in pictures · 2,123 wordsD3Worked examplesEvery scenario covered · 2,371 wordsD4ExercisesGraded, full solutions · 2,157 wordsD5Question bankConcept traps, quick reveals · 2,000 words

Test yourself — Advanced & Emerging Architectures

L1 RecognitionL2 RecallL3 ProductionL4 ApplicationL5 Mastery

Connections

Von Neumann architecture — components, bottleneckCoding · 4.1.1RC circuits — charging, discharging, time constant τ = RCPhysics · 1.8.19