Stiff equations — implicit methods, backward Euler: Visual walkthrough

Before line one, three plain words we will lean on the whole way:

Step 1 — The test problem is one falling curve

WHAT. We study the single equation $y^{'} = λ y .$

WHY. Every method's behaviour on stiff problems is decided by this one line, so we earn it first. Read it aloud: "the slope equals $λ$ times the height." Here $λ$ (Greek "lambda") is just a fixed negative number — a dial. Negative $λ$ means: the higher you are, the faster you fall.

PICTURE. The true motion is $y (t) = y_{0} e^{λ t}$ — a curve that starts at height $y_{0}$ and slides smoothly to $0$ . Make $λ$ more negative and the fall gets brutally fast at the start, then flattens.

Figure — Stiff equations — implicit methods, backward Euler

Step 2 — We cannot follow the curve continuously, so we hop

WHAT. A computer only stores $y$ at discrete instants $t_{0}, t_{1}, t_{2}, \dots$ spaced by a step size $h$ : $t_{n + 1} = t_{n} + h .$

WHY. We must turn the smooth rule $y^{'} = λ y$ into a hopping recipe: given the height now, guess the height one hop later. The only honest tool is the definition of slope — rise over run.

PICTURE. Two dots on the curve, a horizontal gap $h$ apart. The straight line between them has slope "rise ÷ run": $slope \approx \frac{y _{n + 1} - y _{n}}{h} .$ Here $y_{n}$ means "the stored height at hop number $n$ " — just a labelled dot, nothing fancier.

Step 3 — Forward Euler: trust the slope where you stand

WHAT. Set the estimated slope equal to the true slope at the old dot $t_{n}$ : $\frac{y _{n + 1} - y _{n}}{h} = f (t_{n}, y_{n}) = λ y_{n} .$ Solve for the new height: $y_{n + 1} = (1 + hλ) y_{n}$

WHY. It is the cheapest possible move: everything on the right is already known, so $y_{n + 1}$ pops out with no solving. We call it explicit.

PICTURE. Stand on the old dot, read the slope arrow there, and walk in a straight line for a full step $h$ . Because the true curve is bending away from that arrow, a long walk overshoots.

Step 4 — Why forward Euler explodes: the number line of $1 + hλ$

WHAT. Stability means $∣1 + hλ ∣ < 1$ . For real $λ < 0$ write $x = hλ$ (a negative number). We need $- 1 < 1 + x < 1$ , which is $- 2 < x < 0$ . Now translate $x < 0$ back into a bound on $h$ .

WHY the sign flips. We have $- 2 < hλ$ . To isolate $h$ we divide both sides by $λ$ — but $λ$ is negative, and dividing an inequality by a negative number flips the $<$ into $>$ . So $- 2 < hλ$ becomes $\frac{- 2}{λ} > h$ . Since $λ < 0$ , the quantity $\frac{- 2}{λ}$ is positive and equals $\frac{2}{∣ λ ∣}$ . Hence $h < \frac{2}{∣ λ ∣} .$

Why this matters. This inequality is the entire stiffness tragedy in one line. If $λ = - 1000$ , you are chained to $h < 0.002$ forever, long after the curve is flat and boring.

PICTURE. Put $1 + hλ$ on a number line. The safe zone is the segment $(- 1, 1)$ . As $h$ grows, the point $1 + hλ$ marches left, shoots past $- 1$ , and lands where $∣ g_{FE} ∣ > 1$ — every hop now magnifies and flips sign, so the dots ping-pong to infinity.

Step 5 — Backward Euler: trust the slope where you land

WHAT. Same slope bridge, but read the true slope at the new dot $t_{n + 1}$ : $\frac{y _{n + 1} - y _{n}}{h} = f (t_{n + 1}, y_{n + 1}) = λ y_{n + 1} .$

WHY. The new dot is where the curve is calmer (flatter). Aiming at the calm future stops the overshoot. The price: $y_{n + 1}$ now sits on both sides — we must solve for it. That is what implicit means.

PICTURE. Instead of walking off in the old arrow's direction, we ask: "which landing height makes the arrow drawn back from the landing point hit exactly the old dot?" We pick the dot whose own slope points back home.

Step 6 — Why backward Euler is always tame

WHAT. With real $λ < 0$ , the term $- hλ$ is positive, so $1 - hλ > 1$ . Dividing by a number bigger than $1$ shrinks $y$ : $\frac{1}{1 - hλ} < 1 for every h > 0.$

WHY. No matter how greedy the step $h$ , the denominator only grows, so $∣ g_{BE} ∣$ only shrinks. There is no upper limit on $h$ — stability is free.

PICTURE. Plot both growth factors against $h$ (with $λ < 0$ fixed). Forward Euler's $∣1 + hλ ∣$ dives, touches zero, then climbs out of the safe band. Backward Euler's $∣1/ (1 - hλ) ∣$ starts at $1$ and slides gently toward $0$ — never leaving the band.

Step 7 — The full complex picture (every case at once)

WHAT. Real problems have oscillating parts too, so $λ$ can be a complex number (a point in a plane: horizontal axis = decay rate $Re λ$ , vertical = wobble rate $Im λ$ ). To compare against the true solution honestly, give the exact growth per step a name too:

Comparing $∣ g_{true} ∣$ against $∣ g_{FE} ∣, ∣ g_{BE} ∣$ tells us where each method matches reality.

PICTURE.

Forward Euler: $∣1 + hλ ∣ < 1$ is a disc of radius $1$ centred at $- 1$ . Tiny. Most of the left half-plane is outside it — that is instability for stiff $λ$ .
Backward Euler: $∣1/ (1 - hλ) ∣ < 1$ is everything outside the disc of radius $1$ centred at $+ 1$ — which contains the entire left half-plane. That is the definition of A-stable.

Now walk the plane column by column so no $λ$ is left unexplained.

Definition Every column of the plane, spelled out

$Re λ < 0$ (left half — decaying): true solution shrinks ( $∣ g_{true} ∣ = e^{ha} < 1$ ). Backward Euler shrinks too ( $∣ g_{BE} ∣ < 1$ ) for any $h$ , by the box above; forward Euler shrinks only inside its tiny disc.
$Re λ = 0, Im λ \neq = 0$ (imaginary axis — pure oscillation): the true solution $e^{iω t}$ neither grows nor decays, it spins with $∣ g_{true} ∣ = 1$ . Backward Euler gives $∣ g_{BE} ∣ = \frac{1}{1 - ihω} = \frac{1}{1 + ( hω ) ^{2}} < 1$ — it damps the oscillation instead of preserving it. Good for stiff safety, but a warning: pure oscillators lose amplitude they should keep. (The trapezoidal method fixes exactly this, giving $∣ g ∣ = 1$ on the axis.)
$Re λ > 0$ (right half — genuinely blowing up): the true solution $e^{λ t}$ grows without bound; a faithful method should grow too. Forward Euler always does: $∣1 + hλ ∣ > 1$ for real $λ > 0$ . Backward Euler is subtler — for real $λ > 0$ , $∣ g_{BE} ∣ = \frac{1}{∣1 - hλ ∣}$ , so it grows only while $0 < hλ < 2$ ; at $hλ = 2$ the denominator's magnitude is $1$ and $∣ g_{BE} ∣ = 1$ , and for $hλ > 2$ we get $∣1 - hλ ∣ > 1$ , making $∣ g_{BE} ∣ < 1$ — backward Euler then wrongly damps a truly exploding solution. This is the honest small print of A-stability: it guarantees good behaviour for decaying $λ$ , and buys that by not faithfully reproducing growth for large-step unstable $λ$ .

Definition Degenerate & limiting cases (never skipped)

$λ = 0$ (no decay, no wobble): $g_{FE} = 1$ , $g_{BE} = 1$ , and $g_{true} = e^{0} = 1$ . All sit on the boundary — height stays constant, correct.
$h \to \infty$ (backward Euler, $Re λ < 0$ ): $g_{BE} = \frac{1}{1 - hλ} \to 0$ . It doesn't just stay bounded, it snaps to zero — this stronger property is L-stability (see A-stability and L-stability).
Boundary $∣ g_{BE} ∣ = 1$ : this is the whole circle $∣1 - hλ ∣ = 1$ (the unit circle centred at $+ 1$ ), not a single spot. It is hit, for example, at the real point $hλ = 2$ and, in the limit $hω \to 0$ , on the imaginary axis. Everywhere outside that circle (which includes all of the left half-plane) backward Euler is strictly tame.
Nonlinear $f$ : the implicit line can't be rearranged; solve $g (y) = y - y_{n} - h f (t_{n + 1}, y) = 0$ with Newton's method.

The one-picture summary

Two arrows, one plane. Forward Euler walks off the old slope and can leap out of its tiny safe disc. Backward Euler solves for the future dot and its safe region swallows the whole decaying half-plane.

Recall Feynman: the whole walkthrough in plain words

We have a dot that wants to fall to the floor, and the higher it is the faster it falls — that's $y^{'} = λ y$ . A computer can't watch it fall smoothly, so it takes hops of width $h$ and guesses the next height using "rise over run." Forward Euler reads how fast the dot is falling right where it stands and walks straight for a whole hop — but the real fall curves away, so a long hop overshoots, and for a fast faller ( $λ$ very negative) it overshoots so hard it flips sign and blows up unless the hop is microscopic. Backward Euler is the clever cousin: it asks "where should I land so that the fall-speed drawn back from the landing spot points right at where I started?" Because it aims at the calmer future, it never overshoots — the multiplier $1/ (1 - hλ)$ is always a shrink, for any hop size when $λ$ decays. The reason is pure geometry: the number $1 - hλ$ always lands to the right of $1$ when $λ$ decays, so its size beats $1$ and dividing by it shrinks. If the true motion is a pure circle-spin (imaginary $λ$ ), backward Euler still shrinks it a little — it damps wobbles it "should" keep, a small price for safety. And if the true motion genuinely explodes (right-half $λ$ ), forward Euler explodes with it, while backward Euler explodes only for modest steps and then, for very big steps, quietly damps the explosion — the honest cost of being unconditionally safe on the decaying side. In the map of all possible fallers, forward Euler's safe zone is a tiny circle, while backward Euler's safe zone covers the entire land of decaying things. Same accuracy ( $O (h)$ ), wildly better stability — and that is the whole point of going implicit.

Connections

Parent: Stiff equations & backward Euler
Forward Euler method
Region of absolute stability
A-stability and L-stability
Trapezoidal / Crank–Nicolson method
Newton's method for root finding
Runge–Kutta methods
Eigenvalues and time scales of linear systems