Intuition The big pictureA derivative is a limit of a slope: f ′ ( x ) = lim h → 0 f ( x + h ) − f ( x ) h f'(x)=\lim_{h\to0}\frac{f(x+h)-f(x)}{h} f ′ ( x ) = lim h → 0 h f ( x + h ) − f ( x ) h → 0 h\to 0 h → 0 stop early at a small but finite h h h approximation .
Forward : look at where the function is going (uses x x x x + h x+h x + h Backward : look at where it came from (uses x − h x-h x − h x x x Central : average both views → it sees the curvature symmetrically and is the most accurate for the same h h h
WHAT are we doing? Estimating f ′ ( x ) f'(x) f ′ ( x ) table of values or a function we can sample, without an exact symbolic derivative.
WHY does it matter? In real problems f f f
HOW do we get the formulas? From the Taylor series . Everything below is derived , never dumped.
Taylor expansion about x x x ± h \pm h ± h
f ( x + h ) = f ( x ) + h f ′ ( x ) + h 2 2 f ′ ′ ( x ) + h 3 6 f ′ ′ ′ ( x ) + ⋯ f(x+h)=f(x)+hf'(x)+\frac{h^2}{2}f''(x)+\frac{h^3}{6}f'''(x)+\cdots f ( x + h ) = f ( x ) + h f ′ ( x ) + 2 h 2 f ′′ ( x ) + 6 h 3 f ′′′ ( x ) + ⋯ f ( x − h ) = f ( x ) − h f ′ ( x ) + h 2 2 f ′ ′ ( x ) − h 3 6 f ′ ′ ′ ( x ) + ⋯ f(x-h)=f(x)-hf'(x)+\frac{h^2}{2}f''(x)-\frac{h^3}{6}f'''(x)+\cdots f ( x − h ) = f ( x ) − h f ′ ( x ) + 2 h 2 f ′′ ( x ) − 6 h 3 f ′′′ ( x ) + ⋯
Why this step? We want f ′ ( x ) f'(x) f ′ ( x ) f ′ ( x ) f'(x) f ′ ( x )
f ( x + h ) − f ( x ) = h f ′ ( x ) + h 2 2 f ′ ′ ( x ) + ⋯ f(x+h)-f(x)=hf'(x)+\frac{h^2}{2}f''(x)+\cdots f ( x + h ) − f ( x ) = h f ′ ( x ) + 2 h 2 f ′′ ( x ) + ⋯ h h h f ′ ( x ) = f ( x + h ) − f ( x ) h − h 2 f ′ ′ ( ξ ) ⏟ error f'(x)=\frac{f(x+h)-f(x)}{h}-\underbrace{\frac{h}{2}f''(\xi)}_{\text{error}} f ′ ( x ) = h f ( x + h ) − f ( x ) − error 2 h f ′′ ( ξ )
Why this step? Same idea but solve the f ( x − h ) f(x-h) f ( x − h )
f ( x ) − f ( x − h ) = h f ′ ( x ) − h 2 2 f ′ ′ ( x ) + ⋯ f(x)-f(x-h)=hf'(x)-\frac{h^2}{2}f''(x)+\cdots f ( x ) − f ( x − h ) = h f ′ ( x ) − 2 h 2 f ′′ ( x ) + ⋯ f ′ ( x ) = f ( x ) − f ( x − h ) h + h 2 f ′ ′ ( ξ ) f'(x)=\frac{f(x)-f(x-h)}{h}+\frac{h}{2}f''(\xi) f ′ ( x ) = h f ( x ) − f ( x − h ) + 2 h f ′′ ( ξ )
Why this step? Subtract the two expansions so the symmetric f ′ ′ f'' f ′′
f ( x + h ) − f ( x − h ) = 2 h f ′ ( x ) + 2 h 3 6 f ′ ′ ′ ( x ) + ⋯ f(x+h)-f(x-h)=2hf'(x)+\frac{2h^3}{6}f'''(x)+\cdots f ( x + h ) − f ( x − h ) = 2 h f ′ ( x ) + 6 2 h 3 f ′′′ ( x ) + ⋯ h 2 2 f ′ ′ \frac{h^2}{2}f'' 2 h 2 f ′′ die . Divide by 2 h 2h 2 h
f ′ ( x ) = f ( x + h ) − f ( x − h ) 2 h − h 2 6 f ′ ′ ′ ( ξ ) f'(x)=\frac{f(x+h)-f(x-h)}{2h}-\frac{h^2}{6}f'''(\xi) f ′ ( x ) = 2 h f ( x + h ) − f ( x − h ) − 6 h 2 f ′′′ ( ξ )
Why this step? Add the two expansions so the odd (f ′ f' f ′ f ′ ′ ′ f''' f ′′′ f ′ ′ f'' f ′′
f ( x + h ) + f ( x − h ) = 2 f ( x ) + h 2 f ′ ′ ( x ) + h 4 12 f ′ ′ ′ ′ ( x ) + ⋯ f(x+h)+f(x-h)=2f(x)+h^2 f''(x)+\frac{h^4}{12}f''''(x)+\cdots f ( x + h ) + f ( x − h ) = 2 f ( x ) + h 2 f ′′ ( x ) + 12 h 4 f ′′′′ ( x ) + ⋯ f ′ ′ f'' f ′′
Worked example Example 1 — Compare all three onf ( x ) = sin x f(x)=\sin x f ( x ) = sin x x = 1 x=1 x = 1 h = 0.1 h=0.1 h = 0.1 f ′ ( 1 ) = cos 1 = 0.5403023 f'(1)=\cos 1 = 0.5403023 f ′ ( 1 ) = cos 1 = 0.5403023 sin ( 0.9 ) = 0.7833269 , sin ( 1.0 ) = 0.8414710 , sin ( 1.1 ) = 0.8912074 \sin(0.9)=0.7833269,\ \sin(1.0)=0.8414710,\ \sin(1.1)=0.8912074 sin ( 0.9 ) = 0.7833269 , sin ( 1.0 ) = 0.8414710 , sin ( 1.1 ) = 0.8912074
Forward : 0.8912074 − 0.8414710 0.1 = 0.497364 \dfrac{0.8912074-0.8414710}{0.1}=0.497364 0.1 0.8912074 − 0.8414710 = 0.497364 0.0429 0.0429 0.0429 Why this step? Uses x x x x + h x+h x + h ∼ h 2 ∣ f ′ ′ ∣ = 0.1 2 sin 1 ≈ 0.042 \sim \frac h2|f''|=\frac{0.1}{2}\sin 1\approx0.042 ∼ 2 h ∣ f ′′ ∣ = 2 0.1 sin 1 ≈ 0.042
Backward : 0.8414710 − 0.7833269 0.1 = 0.581441 \dfrac{0.8414710-0.7833269}{0.1}=0.581441 0.1 0.8414710 − 0.7833269 = 0.581441 0.0411 0.0411 0.0411
Central : 0.8912074 − 0.7833269 0.2 = 0.539403 \dfrac{0.8912074-0.7833269}{0.2}=0.539403 0.2 0.8912074 − 0.7833269 = 0.539403 0.00090 0.00090 0.00090 Why this step? Average of forward & backward cancels the O ( h ) O(h) O ( h ) ∼ h 2 6 ∣ f ′ ′ ′ ∣ = 0.01 6 cos 1 ≈ 0.0009 \sim\frac{h^2}{6}|f'''|=\frac{0.01}{6}\cos1\approx0.0009 ∼ 6 h 2 ∣ f ′′′ ∣ = 6 0.01 cos 1 ≈ 0.0009
Lesson: central error is ~47× smaller for the same h h h
Worked example Example 2 — Second derivative off ( x ) = e x f(x)=e^x f ( x ) = e x x = 0 x=0 x = 0 h = 0.1 h=0.1 h = 0.1 f ′ ′ ( 0 ) = e 0 = 1 f''(0)=e^0=1 f ′′ ( 0 ) = e 0 = 1 e 0.1 = 1.105171 , e 0 = 1 , e − 0.1 = 0.904837 e^{0.1}=1.105171,\ e^0=1,\ e^{-0.1}=0.904837 e 0.1 = 1.105171 , e 0 = 1 , e − 0.1 = 0.904837 f ′ ′ ( 0 ) ≈ 1.105171 − 2 ( 1 ) + 0.904837 0.01 = 0.010008 0.01 = 1.0008 f''(0)\approx\frac{1.105171-2(1)+0.904837}{0.01}=\frac{0.010008}{0.01}=1.0008 f ′′ ( 0 ) ≈ 0.01 1.105171 − 2 ( 1 ) + 0.904837 = 0.01 0.010008 = 1.0008 Why this step? The combination f ( x + h ) − 2 f ( x ) + f ( x − h ) f(x+h)-2f(x)+f(x-h) f ( x + h ) − 2 f ( x ) + f ( x − h ) ∼ h 2 12 f ′ ′ ′ ′ = 0.01 12 ≈ 0.0008 \sim\frac{h^2}{12}f''''=\frac{0.01}{12}\approx0.0008 ∼ 12 h 2 f ′′′′ = 12 0.01 ≈ 0.0008
Worked example Example 3 — From a raw table (no formula known)
x x x 2.0
2.1
2.2
f f f 0.6931
0.7419
0.7885
Estimate f ′ ( 2.1 ) f'(2.1) f ′ ( 2.1 ) h = 0.1 h=0.1 h = 0.1 0.7885 − 0.6931 0.2 = 0.0954 0.2 = 0.477 \dfrac{0.7885-0.6931}{0.2}=\dfrac{0.0954}{0.2}=0.477 0.2 0.7885 − 0.6931 = 0.2 0.0954 = 0.477 Why this step? We pick the central node 2.1 so we can use both neighbours. (This data is ln x \ln x ln x f ′ ( 2.1 ) = 1 / 2.1 = 0.4762 f'(2.1)=1/2.1=0.4762 f ′ ( 2.1 ) = 1/2.1 = 0.4762
h h h Why it feels right: the math error → 0 \to0 → 0 h → 0 h\to0 h → 0 h h h The fix: the numerator is a difference of nearly-equal numbers → catastrophic cancellation and roundoff blow up like ε / h \varepsilon/h ε / h ≈ h 2 6 ∣ f ′ ′ ′ ∣ ⏟ truncation ↓ + ε h ⏟ roundoff ↑ \approx \underbrace{\tfrac{h^2}{6}|f'''|}_{\text{truncation}\downarrow}+\underbrace{\tfrac{\varepsilon}{h}}_{\text{roundoff}\uparrow} ≈ truncation ↓ 6 h 2 ∣ f ′′′ ∣ + roundoff ↑ h ε optimal h h h ε 1 / 3 \varepsilon^{1/3} ε 1/3
Common mistake Dividing the central formula byh h h 2 h 2h 2 h Why it feels right: forward/backward divide by h h h h h h The fix: central spans from x − h x-h x − h x + h x+h x + h 2 h 2h 2 h halves your answer.
Common mistake Using forward difference at an interior point when you have both neighbours.Why it feels right: forward is the first formula you learn.
The fix: if both x ± h x\pm h x ± h central is one order more accurate for free. Use forward/backward only at the edges of a table where a neighbour is missing.
Common mistake Treating the error symbolO ( h ) O(h) O ( h ) O ( h 2 ) O(h^2) O ( h 2 ) Why it feels right: both shrink as h → 0 h\to0 h → 0 The fix: halving h h h 4 . Over many steps this is the difference between a usable and a useless answer.
Recall The 20% that gives 80%
Central 1st: f ( x + h ) − f ( x − h ) 2 h \dfrac{f(x+h)-f(x-h)}{2h} 2 h f ( x + h ) − f ( x − h ) O ( h 2 ) O(h^2) O ( h 2 )
Central 2nd: f ( x + h ) − 2 f ( x ) + f ( x − h ) h 2 \dfrac{f(x+h)-2f(x)+f(x-h)}{h^2} h 2 f ( x + h ) − 2 f ( x ) + f ( x − h ) O ( h 2 ) O(h^2) O ( h 2 )
Forward/backward O ( h ) O(h) O ( h )
All come from Taylor series : subtract → odd derivative, add → even derivative.
Recall Feynman: explain to a 12-year-oldImagine you're on a hill and want to know how steep it is right where you stand.
Forward : you take one step forward , see how much higher/lower you got, that's your guess of the steepness.Backward : same but you step back .Central : you check one step forward AND one step back, and use the difference between those two spots. Because you peeked on both sides, your steepness guess is much more honest — the little bumps cancel out.
If you take giant steps (h h h insanely tiny steps your ruler isn't precise enough and the answer gets noisy. So pick a sensibly small step.
Mnemonic Remember the cancellation rule"SubtractOdd, AddEven" — Sub tract Taylor expansions to keep odd derivatives (f ′ f' f ′ Add them to keep even derivatives (f ′ ′ f'' f ′′ C entral = C areful with the 2 h.
Forward difference formula for f ′ ( x ) f'(x) f ′ ( x ) f ( x + h ) − f ( x ) h \dfrac{f(x+h)-f(x)}{h} h f ( x + h ) − f ( x ) , error
O ( h ) O(h) O ( h ) .
Backward difference formula for f ′ ( x ) f'(x) f ′ ( x ) f ( x ) − f ( x − h ) h \dfrac{f(x)-f(x-h)}{h} h f ( x ) − f ( x − h ) , error
O ( h ) O(h) O ( h ) .
Central difference for f ′ ( x ) f'(x) f ′ ( x ) f ( x + h ) − f ( x − h ) 2 h \dfrac{f(x+h)-f(x-h)}{2h} 2 h f ( x + h ) − f ( x − h ) , error
O ( h 2 ) O(h^2) O ( h 2 ) .
Central second-derivative formula? f ( x + h ) − 2 f ( x ) + f ( x − h ) h 2 \dfrac{f(x+h)-2f(x)+f(x-h)}{h^2} h 2 f ( x + h ) − 2 f ( x ) + f ( x − h ) , error
O ( h 2 ) O(h^2) O ( h 2 ) .
Why is central more accurate than forward? Subtracting the two Taylor expansions cancels the
f ′ ′ f'' f ′′ term, removing the
O ( h ) O(h) O ( h ) error, leaving
O ( h 2 ) O(h^2) O ( h 2 ) .
What leading error term does forward difference have? − h 2 f ′ ′ ( ξ ) -\dfrac{h}{2}f''(\xi) − 2 h f ′′ ( ξ ) .
What leading error term does central first-difference have? − h 2 6 f ′ ′ ′ ( ξ ) -\dfrac{h^2}{6}f'''(\xi) − 6 h 2 f ′′′ ( ξ ) .
Why can't you make h h h Roundoff/cancellation error grows like
ε / h \varepsilon/h ε / h ; total error has an optimum at finite
h h h .
Trick to derive an even-derivative formula? ADD the
f ( x + h ) f(x+h) f ( x + h ) and
f ( x − h ) f(x-h) f ( x − h ) Taylor series (odd terms cancel).
When must you use forward or backward instead of central? At the edges of a data table where one neighbour is missing.
Finite step h approximation
Central second derivative
O of h squared second order
Intuition Hinglish mein samjhoDekho, derivative ka matlab hai slope — kisi point pe function kitni tezi se badal raha hai. Asli formula mein hum h → 0 h\to0 h → 0 h h h Forward difference aage ka point use karta hai (x x x x + h x+h x + h backward peeche ka (x − h x-h x − h x x x central dono side dekh ke average jaisa kaam karta hai (x − h x-h x − h x + h x+h x + h
Sabse important baat: ye saare formula Taylor series se nikalte hain — koi rata-rataya formula nahi hai. Jab tum f ( x + h ) f(x+h) f ( x + h ) f ( x − h ) f(x-h) f ( x − h ) subtract karte ho, to f ′ ′ f'' f ′′ O ( h 2 ) O(h^2) O ( h 2 ) O ( h ) O(h) O ( h ) h h h add karoge, to odd terms cancel ho jate hain aur tumhe f ′ ′ f'' f ′′
Ek bada trap: log sochte hain ki h h h h h h ε / h \varepsilon/h ε / h optimal h h h
