Numerical differentiation — forward, backward, central differences: Question bank

True or false — justify

Central difference is always more accurate than forward difference for the same

h

False in general. Its truncation error is one order smaller (

O (h^{2})

O (h)

), so it wins for smooth

f

and moderate

h

; but if

f^{'''}

is huge or

h

is in the roundoff-dominated regime, the ordering can break.

The central formula uses the value

f (x)

itself.

False. Central first-derivative

\frac{f ( x + h ) - f ( x - h )}{2 h}

uses only the two neighbours; the midpoint value cancels out. The central second derivative does use

f (x)

Halving

h

roughly quarters the error of the central first-derivative formula.

True. Its leading error is

\frac{h ^{2}}{6} f^{'''}

, and

(h /2)^{2} = h^{2} /4

, so the truncation error drops by about

4 \times

— provided you haven't hit the roundoff floor.

Forward and backward differences have the same order of accuracy.

True. Both are

O (h)

; their leading error terms are

- \frac{h}{2} f^{''}

and

+ \frac{h}{2} f^{''}

— same magnitude, opposite sign. That opposite sign is exactly why averaging them gives central.

The second-derivative formula

\frac{f ( x + h ) - 2 f ( x ) + f ( x - h )}{h ^{2}}

O (h)

because it divides by

h^{2}

False. Order comes from the surviving Taylor term, not the denominator. Adding the expansions kills

f^{'}

and

f^{'''}

, leaving an

\frac{h ^{2}}{12} f^{''''}

error →

O (h^{2})

f

is a straight line, all three first-derivative formulas give the exact slope.

True. For a linear

f

f^{''} = f^{'''} = 0

, so every error term vanishes; each formula returns the exact constant slope regardless of

h

Making

h

smaller can make the answer worse.

True. Below a certain

h

, the numerator subtracts nearly equal numbers → catastrophic cancellation, and roundoff error

\sim ε / h

grows. Total error has a minimum at finite

h

Central difference works fine right at the first row of a data table.

False. It needs both

x - h

and

x + h

. At the first row the left neighbour is missing, so you must fall back to forward difference (or a one-sided higher-order formula).

Spot the error

A student writes central difference as

\frac{f ( x + h ) - f ( x - h )}{h}

. What's wrong?

The span from

x - h

x + h

has width

== 2 h ==

, not

h

. Dividing by

h

instead of

2 h

doubles the answer.

A student computes forward difference with the point behind:

\frac{f ( x ) - f ( x - h )}{h}

and calls it forward.

That is the backward difference. Forward looks ahead using

f (x + h) - f (x)

; this one looks behind.

To get

f^{''}

, a student subtracts the two Taylor expansions instead of adding. What breaks?

Subtracting cancels the even terms (

f^{''}

dies), leaving

f^{'}

— that gives you the first derivative. You must add to keep the even

f^{''}

term.

A student claims the forward-difference error is

- \frac{h ^{2}}{2} f^{''}

. Fix it.

The error is

- \frac{h}{2} f^{''}

, first power of

h

(that's why it's

O (h)

). The

h^{2}

belongs to the central first-derivative error, which carries

f^{'''}

, not

f^{''}

Someone reads "

O (h^{2})

" as "the error equals

h^{2}

". Why is that wrong?

O (h^{2})

means the error is proportional to

h^{2}

up to a constant times a derivative of

f

; the actual size is

\approx \frac{h ^{2}}{6} ∣ f^{'''} ∣

. See Truncation error and Big-O.

A student uses

h = 1 0^{- 14}

in double precision "to be safe" and gets garbage. Diagnose.

At that

h

f (x + h)

and

f (x - h)

agree to ~15 digits, so their difference is nearly all roundoff. The

ε / h

term explodes. See Roundoff error and floating point.

Why questions

Why do the two Taylor expansions cancel the

f^{''}

term when we subtract them?

f (x - h)

the even-power terms keep the same sign as in

f (x + h)

, so subtracting removes them, while odd-power terms (

f^{'}

f^{'''}

) flip sign and reinforce. Even ↔ symmetric, odd ↔ antisymmetric.

Why is central difference "one extra order of accuracy for free"?

The symmetry does the work: by sampling equally on both sides, the leading

O (h)

curvature error from the two sides is equal and opposite and cancels, so the first surviving term is

O (h^{2})

— no extra function evaluations needed.

Why can't a computer just take

h \to 0

like the limit definition demands?

A machine stores only finite floating-point numbers with a smallest gap

ε

; there are no infinitesimals. Beyond some tiny

h

, subtraction loses all significant digits.

Why does the optimal

h

for central difference scale like

ε^{1/3}

Total error

\approx \frac{h ^{2}}{6} ∣ f^{'''} ∣ + \frac{ε}{h}

; minimising the sum (balancing a rising

ε / h

against a falling

h^{2}

) sets the derivative to zero, giving

h_{opt} \propto ε^{1/3}

Why prefer central difference at interior table points but not at the edges?

Interior points have both neighbours available, so central gives

O (h^{2})

for free; edges lack one neighbour, forcing a one-sided (

O (h)

) formula unless you reach further in for a higher-order stencil.

Why does adding the expansions leave a denominator

h^{2}

for

f^{''}

but

2 h

for

f^{'}

The surviving term after adding is

h^{2} f^{''}

, so you divide by

h^{2}

to isolate

f^{''}

; after subtracting the surviving term is

2 h f^{'}

, so you divide by

2 h

. The denominator just matches whatever power of

h

multiplies the target derivative.

Why is Richardson extrapolation a natural next step after central differences?

Because the error is a clean power series in

h^{2}

, combining results at

h

and

h /2

can algebraically cancel the leading

h^{2}

term, boosting accuracy to

O (h^{4})

without new derivatives.

Edge cases

What does the central second-derivative formula give if

f

is exactly a parabola?

The exact

f^{''}

, for any

h

. All terms from

f^{''''}

onward are zero for a quadratic, so no truncation error remains.

You only have a data table (no formula) — can you still choose

h

small for accuracy?

No.

h

is fixed by the table's spacing; you cannot sample between rows. You can only choose which stencil (forward/central) and whether to interpolate first.

Node spacing in your table is uneven (

x - h_{1}, x, x + h_{2}

with

h_{1} \neq = h_{2}

). Does the standard central formula still hold?

No — the clean

O (h^{2})

cancellation assumed symmetric spacing. Unequal spacing needs a general finite-difference weight derivation, and the naive formula loses an order.

What happens to forward difference at a kink (point where

f^{'}

is discontinuous)?

Taylor's theorem needs

f

smooth enough; at a kink the expansion is invalid, so the error estimates break and the formula silently returns a slope that averages across the corner.

f (x)

is constant, what does each formula return?

Exactly

0

for the first derivative and

0

for the second — all differences of equal values vanish, matching the true zero derivative for any

h

h

so large that

x + h

leaves the region where

f

is smooth, is central still "best"?

Not necessarily. The

O (h^{2})

label only describes the leading term as

h \to 0

; for large

h

higher-order terms dominate and the accuracy comparison is no longer guaranteed.

Can the second-derivative formula ever give a negative value where the true

f^{''}

is positive?

Yes — near the roundoff floor, cancellation noise in

f (x + h) - 2 f (x) + f (x - h)

can flip the tiny result's sign entirely, a classic symptom of

h

being too small.

Connections

Taylor series — the source of every cancellation argument above.
Truncation error and Big-O — what $O (h)$ vs $O (h^{2})$ actually claims.
Roundoff error and floating point — the $ε / h$ floor behind the "smaller isn't better" traps.
Richardson extrapolation — exploits the $h^{2}$ error structure asked about above.
Finite difference methods for PDEs — where uneven grids and edge stencils really bite.