Taylor series approximation: Visual walkthrough

We use one running example so every picture is comparable: the real function $f (x) = e^{x}, a = 0.$ We pick $e^{x}$ because its derivatives are the friendliest possible ( $f^{'} (x) = e^{x}$ , so $f^{(n)} (0) = 1$ for every $n$ ), which lets the shape of the argument shine without messy arithmetic hiding it.

Before we start — the words we will use

Our plan: build a polynomial $P (x) = c_{0} + c_{1} (x - a) + c_{2} (x - a)^{2} + c_{3} (x - a)^{3} + \dots$ and choose the numbers $c_{0}, c_{1}, c_{2}, \dots$ one at a time so that $P$ matches $f$ 's value, then slope, then curvature, then higher bends. Why $(x - a)$ and not just $x$ ? Because measuring distance from the anchor point $a$ makes every term except the one we want vanish when we set $x = a$ — you'll see that trick fire in every step.

Step 1 — Match the value (the order-0 picture)

WHAT. Set $x = a$ in $P$ . Every term containing $(x - a)$ becomes $(a - a) = 0$ and dies. Only the constant survives: $P (a) = c_{0} .$ We demand $P (a) = f (a)$ , so $c_{0} = f (a) .$

WHY. The height at $a$ is controlled by $c_{0}$ alone — nothing else can touch it once every other term is switched off. So the very first fact we lock in is "start at the right height."

PICTURE. Our approximation is a flat horizontal line at height $f (0) = 1$ . It touches $e^{x}$ at the anchor dot but immediately drifts away — a value match and nothing more.

In the equation $P_{0} (x) = height at a c_{0}$ : the lone constant is the whole approximation so far.

Step 2 — Match the slope (the order-1 picture)

WHAT. Differentiate $P$ once: $P^{'} (x) = c_{1} + 2 c_{2} (x - a) + 3 c_{3} (x - a)^{2} + \dots$ Now set $x = a$ : again every $(x - a)$ term dies, leaving $P^{'} (a) = c_{1}$ . Demand $P^{'} (a) = f^{'} (a)$ : $c_{1} = f^{'} (a) .$

WHY. A flat line gets the height right but points the wrong way. Matching the slope tilts our line so it touches tangentially instead of crossing — the approximation now agrees with $f$ to first order. This is exactly the picture Gradient Descent lives in: "know the value and the slope, step downhill."

PICTURE. The horizontal line rotates into the tangent line $1 + x$ . Notice it stays glued to $e^{x}$ over a much wider stretch than the flat line did.

Term by term $P_{1} (x) = height c_{0} + slope c_{1} (x - a)$ : the new piece $c_{1} (x - a)$ is a straight ramp whose steepness is the slope and whose value is $0$ at $a$ — so it adjusts the tilt without disturbing the height we already fixed.

Step 3 — Where the factorial is born (match the curvature)

WHAT. Differentiate once more: $P^{''} (x) = 2 c_{2} + 6 c_{3} (x - a) + \dots$ Set $x = a$ : $P^{''} (a) = 2 c_{2}$ . Demand $P^{''} (a) = f^{''} (a)$ : $2 c_{2} = f^{''} (a) \Rightarrow c_{2} = \frac{f ^{''} ( a )}{2} .$

WHY. Here is the moment the factorial sneaks in. The term $c_{2} (x - a)^{2}$ , differentiated twice, produces a $2$ (from $2 \cdot 1$ ). To get the curvature to come out equal to $f^{''} (a)$ we must divide by that $2$ to cancel it. The $2$ is not a fudge factor — it is literally $2!$ , the number $(x - a)^{2}$ spits out.

PICTURE. The straight tangent line curls into a parabola $1 + x + \frac{x ^{2}}{2}$ that now bends the same way as $e^{x}$ . This is the exact picture Newton's Method uses: fit a parabola, jump to where its curvature says the bottom is.

$P_{2} (x) = value c_{0} + tilt c_{1} (x - a) + bend \frac{f ^{''} ( a )}{2} (x - a)^{2}$ : the new $(x - a)^{2}$ piece is $0$ and has zero slope at $a$ , so it fixes the bend without disturbing the value or slope already locked in. Each new term only touches the derivative it is responsible for.

Step 4 — See the general pattern

WHAT. Push the same idea to the $n$ -th term $c_{n} (x - a)^{n}$ . Differentiate it $n$ times. Each differentiation peels off one power and multiplies by the exponent: $(x - a)^{n} \frac{d}{d x} n (x - a)^{n - 1} \frac{d}{d x} n (n - 1) (x - a)^{n - 2} \to \dots \to n (n - 1) \dots 1 = n! .$ At $x = a$ all lower terms have already vanished and all higher terms still carry a factor $(x - a)$ . So $P^{(n)} (a) = n! c_{n} \Rightarrow c_{n} = \frac{f ^{(n)} ( a )}{n !} .$

WHY. The $n!$ is the accumulated stack of exponents you unavoidably pick up while differentiating $(x - a)^{n}$ down to a constant. Dividing by $n!$ undoes exactly that pile-up, so the $n$ -th coefficient carries the pure $n$ -th derivative of $f$ and nothing else.

PICTURE. Watch the exponent countdown $3 \to 2 \to 1$ produce $3 \cdot 2 \cdot 1 = 6 = 3!$ as a staircase of multiplications.

Plugging every $c_{n}$ back gives the full series $f (x) = \sum_{n = 0}^{\infty} coefficient c_{n} \frac{f ^{(n)} ( a )}{n !} shape term (x - a)^{n} .$ We derived it — no memorising — just "demand all derivatives match."

Step 5 — Watch accuracy improve, order by order

WHAT. Stack the approximations from Steps 1–3 plus the order-3 curve $1 + x + \frac{x ^{2}}{2} + \frac{x ^{3}}{6}$ on top of the true $e^{x}$ .

WHY. Seeing them together makes the central claim visible: each extra derivative you match buys accuracy over a wider window. The order-0 line is right at one point, order-1 over a small arc, order-2 over more, order-3 more still.

PICTURE. Four coloured curves fanning out from the same anchor dot, each hugging $e^{x}$ longer than the last.

Step 6 — The degenerate & failure cases (never hit a surprise)

Approximation must survive its own edge cases, so we walk them.

Case A — a term contributes nothing. If $f^{(n)} (a) = 0$ , then $c_{n} = 0$ and that power simply drops out. For $sin x$ at $a = 0$ every even derivative is $0$ (because $sin 0 = 0$ ), so the series is all odd powers: $x - \frac{x ^{3}}{3 !} + \frac{x ^{5}}{5 !} - \dots$ . Nothing breaks; the slot is just empty.

Case B — the series has a finite radius. Not every series matches its function everywhere. For $\frac{1}{1 - x} = 1 + x + x^{2} + \dots$ the pictures agree only for $∣ x ∣ < 1$ ; at $x = 2$ the partial sums blow up while the true curve stays calm. The approximation is local.

Case C — the ultimate warning. $f (x) = e^{- 1/ x^{2}}$ (with $f (0) = 0$ ) has every derivative equal to $0$ at $0$ . So every $c_{n} = 0$ and its Taylor series is the flat line $0$ — yet $f$ itself is not $0$ . Matching all derivatives at one point does not guarantee matching the function away from it.

PICTURE. Left: $1/ (1 - x)$ and its order-6 series diverging past $x = 1$ . Right: $e^{- 1/ x^{2}}$ sitting stubbornly above its zero Taylor series.

The one-picture summary

One frame, the whole story: a flat line (value) tilts into a tangent (slope), curls into a parabola (curvature), then into a cubic — each matching one more derivative of $e^{x}$ at the anchor, each courtesy of the coefficient $c_{n} = f^{(n)} (a) / n!$ whose factorial cancels the $n!$ that differentiation dumps on $(x - a)^{n}$ .

Recall Feynman retelling of the whole walkthrough

I want to fake a curve using only what I know at one spot. First I get the height right — that's a flat line, correct for exactly one point. Then I match how tilted it is — my line rotates to kiss the curve, and now it's right over a little stretch. Then I match how it bends — my line becomes a bowl that hugs the curve over a bigger stretch. Every time I add a fact (the next derivative), I add one more term shaped like $(x - a)^{n}$ , and I have to divide by $n!$ because differentiating that shape $n$ times keeps coughing up a factor of $n!$ that I need to cancel. The neat trick: because each new term is flat-and-level at the anchor, it fixes its own derivative without messing up the ones I already fixed. The catch: this magic only works near the anchor — go too far and the fake curve wanders off, and some sneaky functions ignore the trick entirely.

Recall

Why does setting

x = a

kill every term but

c_{0}

in Step 1?

Each other term has a factor

(x - a)

, which becomes

0

x = a

Where does the factorial come from?

Differentiating

(x - a)^{n}

exactly

n

times multiplies by

n (n - 1) \dots 1 = n!

, and we divide by it to cancel that.

Why does each new

(x - a)^{n}

term not spoil earlier matches?

x = a

that term and its first

n - 1

derivatives are all

0

, so it only affects the

n

-th derivative.

How does the remainder

R_{k}

scale near

a

(x - a)^{k + 1}

— tiny near

a

, and tinier faster for larger

k

What does

e^{- 1/ x^{2}}

warn us about?

All its derivatives at

0

vanish, so its Taylor series is

0

yet the function isn't — smooth

\neq =

equal to its series.

Connections

Derivatives and gradients — every coefficient is a derivative at $a$ .
Gradient Descent — the Step 2 (order-1) picture.
Newton's Method — the Step 3 (order-2 parabola) picture.
Hessian matrix — the curvature term in the multivariable version.
Convex functions — positive curvature everywhere.
Approximation error and Big-O — formalises the Step 5 remainder.
Hinglish version