5.4.22 · D1Scientific Computing (Python)

Foundations — Floating point gotchas — catastrophic cancellation, associativity failure

1,463 words7 min readBack to topic

This page assumes nothing. Before you touch the parent note Floating Point Gotchas, we build every symbol it uses, one brick at a time, each brick resting on the one before it.


0. What even is a "number" to a computer?

A calculator screen has a fixed width. If it can show 4 digits, then fits (as ) but adding changes nothing — the falls off the right edge. A real computer is the same idea with more digits. Everything below is a precise version of "digits fall off the edge."

Figure — Floating point gotchas — catastrophic cancellation, associativity failure

1. Bits, base-2, and the mantissa

A bit is a single 0 or 1. Computers write numbers in base 2 (binary) the same way we write them in base 10 — but the place-values are powers of , not .

In base 10, . In base 2, .

WHY split it this way? Because this is scientific notation in binary. Storing "digits" and "scale" separately lets one small chunk of memory represent both and . The topic needs this because the exponent decides the size of the gap — and gap size is the whole story.


2. The gap, and machine epsilon

Because the mantissa has only 52 bits, near the number the smallest change you can make is flipping the last bit, worth . That is the size of the gap right after .

Figure — Floating point gotchas — catastrophic cancellation, associativity failure

The picture shows the crucial fact: the gap grows with the number's magnitude. Near the gap is ; near the gap is about ; near the number is smaller than one gap and simply vanishes when added. That single visual explains the parent's "" absorption.


3. Relative vs absolute error — the two rulers

WHY two? Being off by is disastrous for (50%) but negligible for (a billionth). The computer's guarantee is stated in the relative ruler because that is what stays constant across all magnitudes.

Figure — Floating point gotchas — catastrophic cancellation, associativity failure

HOW to read the symbols:

  • ::: "float of " — the nearest representable double to .
  • (delta) ::: the tiny relative fudge, somewhere in .
  • ::: because rounding picks the nearer neighbour, you are never off by more than half a gap.

Everything the parent derives — the cancellation amplifier, the summation bound — is algebra applied to this one line.


4. Symbols you will meet in the parent note


5. How these bricks build the topic

Bits and base-2

Mantissa and exponent

The gap grows with magnitude

Machine epsilon

Relative vs absolute error

Rounding model fl x = x times 1 plus delta

Cancellation amplifier

Associativity failure and absorption

Quadratic and 1 minus cos x fixes

Kahan summation

Numerical stability

Read it top-down: raw bits give the format, the format gives the gap, the gap gives , and together with the relative/absolute distinction you get the rounding model — the single equation from which Catastrophic cancellation, Round-off error propagation, Kahan compensated summation and Quadratic formula numerical issues all follow. The whole chain lands in Numerical stability and conditioning.


Equipment checklist

You are ready for the parent note when you can answer each without peeking:

Convert to decimal.
.
How many mantissa bits does an IEEE-754 double store?
52 (plus a hidden leading 1 = 53 bits of precision).
What is and what does it measure?
, the relative gap between and the next double.
Absolute vs relative error — define both for stored of true .
absolute ; relative .
State the rounding model and the meaning of every symbol.
, ; = round to nearest double, = relative slip, = half-gap max.
Why does the gap between representable numbers grow for large numbers?
the exponent scales the whole mantissa, so the last-bit step grows with .
Why is the computer's guarantee stated in relative error?
because it stays constant across all magnitudes, unlike absolute error.