Implementing ODE solvers from scratch — Euler, RK4: Visual walkthrough

Everything here deepens the parent note: the parent topic.

Step 1 — What an ODE actually hands you

WHAT. We do not know the curve $y (t)$ . We only own the machine $f$ that reports the slope at any point we choose to stand.

WHY. For most real problems there is no formula for $y (t)$ — but $f$ is always a thing you can compute (it's just arithmetic). So the game is: turn "I know the slope everywhere" into "I know the curve."

PICTURE. Below, the faint dashed curve is the true $y (t)$ (which the solver cannot see). At one point we place a short arrow — that arrow is what $f$ tells us: the direction to walk right here.

Figure — Implementing ODE solvers from scratch — Euler, RK4

$f (t, y)$ ::: the machine that returns the slope $y^{'}$ at any point $(t, y)$ you stand on.
What smoothness does RK4 need? ::: $f$ smooth enough that $y (t)$ has a valid Taylor expansion through the $h^{4}$ term (roughly 4 continuous derivatives).

Step 2 — Laying down the grid, then turning a slope into a step

WHAT. We refuse to take $h$ all the way to zero. We keep it small but finite and read the equation backwards to solve for the next rung.

WHY this tool. We want to predict $y$ one rung ahead, and the derivative definition is the only bridge between "slope now" and "value a bit later." Dropping the limit is the single approximation that starts everything.

HOW — the algebra, one move at a time. Start from the finite- $h$ version and drop the limit: $f (t_{n}, y_{n}) \approx \frac{y _{n + 1} - y _{n}}{h}$ Multiply both sides by $h$ (clears the fraction): $h f (t_{n}, y_{n}) \approx y_{n + 1} - y_{n}$ Add $y_{n}$ to both sides (isolates the unknown $y_{n + 1}$ ): $y_{n + 1} = y_{n} + h slope at where we stand f (t_{n}, y_{n})$

PICTURE. The right triangle below makes rise-over-run literal: run $= h$ along the bottom, rise $= h \cdot f (t_{n}, y_{n})$ up the side. Follow the slope line and you land at $y_{n + 1}$ .

$h$ ::: the step size — the horizontal gap between grid rungs $t_{n}$ and $t_{n + 1}$ .
$y_{n}$ ::: our current best estimate of $y$ at time $t_{n} = t_{0} + nh$ .
How are the discrete times defined? ::: $t_{n + 1} = t_{n} + h$ , so $t_{n} = t_{0} + nh$ .

Step 3 — Euler's method: trust one slope, take the whole hop

WHAT. We measure the slope once, at the start of the interval, and walk in that straight line for the entire step $h$ .

WHY it's shaky. The true curve bends during the step. But Euler froze the slope at the left edge, so if the curve is concave up, Euler's straight line falls below the truth; concave down, it overshoots above. The error is baked in the moment we commit to a single slope.

PICTURE. Watch the red Euler segment leave the true curve tangentially and then drift away as the curve bends up under it. The vertical gap at $t_{n + 1}$ is the local error of this one step.

Step 4 — The RK4 idea: don't guess, peek ahead

WHAT. We will collect four slope samples $k_{1}, k_{2}, k_{3}, k_{4}$ before committing to a single averaged hop.

WHY four, and why the middle twice. Expand the true step as a Taylor series in $h$ : the exact answer is $y_{n} + h y^{'} + \frac{h ^{2}}{2} y^{''} + \frac{h ^{3}}{6} y^{'''} + \frac{h ^{4}}{24} y^{''''} + \dots$ . Each slope sample $k_{i}$ , itself Taylor-expanded, contributes its own $h, h^{2}, h^{3}, h^{4}$ pieces. The weights $1, 2, 2, 1$ over $6$ are the unique mixture that makes the $h^{1}, h^{2}, h^{3}, h^{4}$ pieces of the blend match the true series term-for-term; the first mismatch is at $h^{5}$ . The two midpoint samples carry the bulk of the $h^{2}$ and $h^{3}$ corrections — the same instinct as Simpson's Rule, which also loads weight on the midpoint.

PICTURE. Left panel: four slope arrows on the interval (start $k_{1}$ , midpoint $k_{2}, k_{3}$ , end $k_{4}$ ). Right panel: a skeletal "ledger" showing each power of $h$ and how the weighted blend cancels the true-series term at that power, leaving the first leftover at $h^{5}$ .

Which power of $h$ is the first RK4 does not match? ::: $h^{5}$ — everything through $h^{4}$ is cancelled exactly.

Step 5 — Building $k_{1}$ and $k_{2}$ (start, then first midpoint peek)

WHAT. $k_{1}$ is our first honest reading. $k_{2}$ uses $k_{1}$ merely to scout the midpoint, then measures a fresh slope at that scouted spot.

WHY. The midpoint slope corrects for the curve's bending that $k_{1}$ alone missed. We must move to the midpoint (in both $t$ and $y$ ) before reading, otherwise we'd just get $k_{1}$ again.

PICTURE. The dashed blue trial-line is the half-step driven by $k_{1}$ ; the solid arrow at its tip is $k_{2}$ , visibly steeper because the curve is climbing.

$k_{1}$ ::: slope at the left edge, the same slope Euler uses.
$k_{2}$ ::: slope at the midpoint reached by a trial half-step along $k_{1}$ .

Step 6 — Refining with $k_{3}$ and reaching the end with $k_{4}$

WHAT. $k_{3}$ is a second, improved midpoint estimate; $k_{4}$ is the end-of-interval estimate built on $k_{3}$ .

WHY the chain $k_{1} \to k_{2} \to k_{3} \to k_{4}$ . Each stage stands on the previous one so that errors partially cancel when averaged. This sequential dependence is the whole trick.

PICTURE. Two midpoint arrows ( $k_{2}$ pale-yellow, $k_{3}$ pink) sit almost on top of each other but the pink $k_{3}$ is nudged closer to truth; the far-right arrow is $k_{4}$ .

Step 7 — The weighted average: one hop worth four peeks

WHAT. We collapse four readings into one averaged slope and take a single Euler-style hop with it.

WHY $1, 2, 2, 1$ . The midpoint best represents the interval's average behaviour, so it earns double weight — echoing how Simpson's Rule loads its middle sample. These exact numbers are the only choice that makes the update agree with the true Taylor series through the $h^{4}$ term (Step 4).

PICTURE. The green averaged slope (blended from the four arrows) drives one clean hop that lands essentially on the true curve — compare the tiny green gap to Step 3's fat red gap.

Step 8 — Edge & degenerate cases (so nothing surprises you)

WHAT. We check the boundaries: an $f$ that ignores $t$ , an $f$ that ignores $y$ , a nearly-flat step, and a backward step.

Autonomous $f (y)$ only (no $t$ ): the $t + \frac{h}{2}$ and $t + h$ shifts still exist but do nothing, since $f$ ignores $t$ . RK4 still works; the shifts are simply harmless — the figure below shows the two midpoint samples landing at the same slope value because only $y$ matters.
Pure-time $f (t)$ only (no $y$ , e.g. $y^{'} = sin t$ ): now every $y + \dots$ inside the stages is ignored, and RK4 collapses exactly into Simpson's rule for $\int f d t$ . Here the two midpoint samples $k_{2}, k_{3}$ become identical — they fuse into Simpson's single midpoint with combined weight $2 + 2 = 4$ . This is the clearest proof of the RK4–Simpson kinship.
Flat step, $f \equiv 0$ : all four $k$ 's are $0$ , so $y_{n + 1} = y_{n}$ . Both methods correctly refuse to move. No division blows up because we divide by the constant $6$ , never by $h$ .
Backward step, $h < 0$ : every formula is unchanged; $t$ and $y$ simply move left. Useful for reversing an integration.

WHY show these. If you only ever test a bending curve you might forget that forgetting the $t$ -shifts silently works for autonomous $f$ and breaks for non-autonomous $f$ .

PICTURE (autonomous). Left: $f$ depends on $y$ only, so at both midpoint times the slope read is the same height-driven value — the $t$ -shift changes nothing.

PICTURE (pure-time → Simpson). Here $f = sin t$ : the four slope readings land at $t_{n}, t_{n} + \frac{h}{2}, t_{n} + h$ ; the two midpoint reads coincide and the shaded area under $sin t$ is what RK4 computes — literally Simpson's rule.

The one-picture summary

WHAT. One figure, both methods, same start, same $h$ : Euler's red straight hop drifting off, RK4's four peeks blending into a green hop that hugs the true curve. Watch the two error gaps at $t_{n + 1}$ .

Recall Feynman: the whole walkthrough in plain words

You're crossing a valley in fog. You mark equally spaced checkpoints ahead — that's the grid $t_{n + 1} = t_{n} + h$ . All you have is a compass that, wherever you stand, tells you which way the ground tilts — that's $f (t, y)$ . Euler reads the tilt once at your feet and marches a full step in that direction. But the valley curves, so you drift off the trail; take smaller steps and you drift less, but you'll be walking forever. RK4 is patient: it reads the tilt at your feet ( $k_{1}$ ), scouts half a step ahead and reads it there ( $k_{2}$ ), scouts the midpoint again using that better reading ( $k_{3}$ ), then scouts the far end ( $k_{4}$ ). It blends the four readings — counting the two middle ones double, dividing by six — into one trustworthy direction, and takes a single step along that. Four careful peeks per step keep you almost perfectly on the trail, which is why serious computing leans on RK4. (Walking backward? Just make $h$ negative — every formula still holds.)

Connections

Taylor Series Expansion — the $1, 2, 2, 1$ weights are forced by matching this series to $h^{4}$ .
Simpson's Rule — Step 8's pure-time case shows RK4 literally becoming Simpson.
Finite Difference Approximation of Derivatives — Step 2's dropped limit is the forward difference.
Numerical Stability and Stiff ODEs — what happens when even RK4's small step isn't safe.
Adaptive Step Size (RK45 / Dormand–Prince) — let the solver choose $h$ from the error it senses.
scipy.integrate.solve_ivp — the production tool that wraps all of this.