Taylor series approximation: Question bank — AI-ML deep dive

Before we start, one word we will lean on constantly: local. A statement is local at $a$ if it only claims to be true for $x$ close to $a$ — a small neighborhood around that one point. Taylor's whole promise is local, and most traps below come from forgetting that.

True or false — justify

A Taylor polynomial of order

k

matches

f

AND its first

k

derivatives at

a

True — that is literally how the coefficients are chosen;

c_{n} = f^{(n)} (a) / n!

is set so the

n

-th derivative of the polynomial equals

f^{(n)} (a)

x = a

for every

n \leq k

If two smooth functions share the same Taylor series at

a

, they are equal near

a

False —

f (x) = e^{- 1/ x^{2}}

(with

f (0) = 0

) and

g (x) = 0

share the all-zeros series at

0

, yet

f \neq = 0

for any

x \neq = 0

. Matching every derivative at one point does not force equality.

The Maclaurin series is a different kind of series from the Taylor series.

False — Maclaurin is just the name for a Taylor series with the special choice

a = 0

; nothing new is happening, only the expansion point moved to the origin.

A convergent Taylor series always converges to the function that generated it.

False — convergence of the series and equality with the function are two separate facts. The

e^{- 1/ x^{2}}

example converges (to

0

) everywhere but equals its function nowhere except the origin.

Adding more Taylor terms always improves accuracy at a fixed point

x

False — only inside the radius of convergence. For

\frac{1}{1 - x}

x = 2

, each extra term makes the partial sum worse because

2^{n}

blows up; more terms help only where the series converges.

The remainder

R_{k}

vanishing as

k \to \infty

is what guarantees the series equals

f

True — equality holds exactly when

R_{k} \to 0

. The Taylor series "works" precisely on the set of

x

where the leftover error can be driven to zero.

For an even function, the Maclaurin series contains only even powers of

x

True — an even function has

f^{'} (0) = f^{'''} (0) = \dots = 0

(all odd derivatives vanish at

0

), so every odd-power coefficient dies, leaving only

x^{0}, x^{2}, x^{4}, \dots

— which is why

cos x

has no odd terms.

A degree-1 Taylor polynomial is always the tangent line to the curve at

a

True — order-1 matches value and slope, and "a line matching height and slope at a point" is the definition of the tangent; higher terms only bend it into the curve.

Spot the error

f (a) + f^{'} (a) (x - a) + f^{''} (a) (x - a)^{2} + f^{'''} (a) (x - a)^{3}

is the 3rd-order Taylor polynomial."

The factorials are missing. Correct is

f (a) + f^{'} (a) (x - a) + \frac{f ^{''} ( a )}{2 !} (x - a)^{2} + \frac{f ^{'''} ( a )}{3 !} (x - a)^{3}

; without dividing by

n!

the higher terms are far too large.

"To approximate

ln x

near

x = 1

, use the Maclaurin series (expand at

0

)."

You cannot expand

ln x

0

— it is undefined there (and its derivatives blow up). Expand at

a = 1

, the point you actually care about, giving

ln x = (x - 1) - \frac{( x - 1 ) ^{2}}{2} + \dots

"The remainder is

\frac{f ^{(k + 1)} ( a )}{( k + 1 )!} (x - a)^{k + 1}

The derivative is evaluated at some unknown

ξ

between

a

and

x

, not at

a

R_{k} = \frac{f ^{(k + 1)} ( ξ )}{( k + 1 )!} (x - a)^{k + 1}

. Using

a

would make

R_{k}

just another Taylor term, not the true error.

"Newton's method step is

Δ = - H \nabla L

It is

Δ = - H^{- 1} \nabla L

— the inverse Hessian. Minimizing the 2nd-order model

\nabla L + H Δ = 0

solves to

Δ = - H^{- 1} \nabla L

; using

H

itself has the wrong units and points the wrong way (see Newton's Method, Hessian matrix).

"Gradient descent uses the second-order Taylor expansion."

Gradient descent uses only the first-order piece

L (θ_{0}) + \nabla L^{⊤} Δ

plus a chosen step size

η

; it deliberately throws away the curvature term

\frac{1}{2} Δ^{⊤} H Δ

, which is what distinguishes it from Newton (see Gradient Descent).

sin x \approx x

is only valid because

sin

is small."

The approximation is valid because the next term is

- x^{3} /6

, which is tiny when $x$ is near $0$ ; it is a statement about

x

being small, not about

sin

. At

x = 3

the approximation

x = 3

badly overshoots

sin 3 \approx 0.14

"Since all derivatives of

e^{x}

equal

e^{x}

, the coefficients of its Maclaurin series grow."

The derivatives evaluated at $0$ all equal

1

, so the coefficients are

1/ n!

, which shrink rapidly. The point of evaluation collapses the growing function to a constant

1

Why questions

Why does a factorial appear in the denominator, not some other number?

Differentiating

(x - a)^{n}

exactly

n

times produces the factor

n! = n \cdot (n - 1) \dots 1

; dividing by

n!

is precisely what cancels it so the polynomial's

n

-th derivative lands on

f^{(n)} (a)

Why does matching more derivatives improve the approximation near

a

The remainder scales like

(x - a)^{k + 1}

; raising the order

k

raises that exponent, so for

∣ x - a ∣ < 1

the error term

(x - a)^{k + 1}

shrinks much faster (see Approximation error and Big-O).

Why do ML optimizers care about Taylor series at all?

A loss surface is expensive and complicated, but locally it is a simple polynomial in the step

Δ

; optimizers minimize that cheap local model instead — 1st-order gives gradient descent, 2nd-order gives Newton.

Why must the curvature term use

H ⪰ 0

(a positive-semidefinite Hessian) for a convex bowl?

The 2nd-order term

\frac{1}{2} Δ^{⊤} H Δ

is the local shape; if

H ⪰ 0

it is a bowl opening upward in every direction, which is exactly the local condition for Convex functions and guarantees the Newton step aims at a minimum, not a saddle.

Why is expanding at the right point

a

more important than adding more terms?

Accuracy is governed by

(x - a)^{k + 1}

; if

∣ x - a ∣

is large no number of terms may converge, but shrinking

∣ x - a ∣

by choosing

a

near your target improves every order at once.

Why does the Taylor polynomial's error term evaluate the derivative at a mystery point

ξ

Because the true function bends by an amount somewhere between its bend at

a

and at

x

; the Mean-Value-style

ξ

captures "some representative curvature in between" without us needing to know exactly where.

Why do only odd powers survive in the series for

sin x

a = 0

the even-order derivatives of

sin

are

\pm sin 0 = 0

, so their coefficients vanish, leaving

x - x^{3} /3! + x^{5} /5! - \dots

— the series inherits

sin

's odd symmetry.

Edge cases

What is the Taylor series of a polynomial (say degree 3) — does it ever "approximate"?

It reproduces the polynomial exactly: all derivatives above order 3 are zero, so the series truncates and the remainder is identically

0

. Taylor of a polynomial is not an approximation, it is an equality everywhere.

What happens to the Taylor series at

x = a

itself?

Every term with

(x - a)

vanishes, leaving only

c_{0} = f (a)

; the approximation is exact at the expansion point, with zero error, no matter the order.

Can a function be infinitely differentiable yet have a useless Taylor series?

Yes —

e^{- 1/ x^{2}}

0

is smooth with every derivative zero there, so its series is

0

and captures nothing about the true (nonzero) function; smoothness does not guarantee a helpful series.

What does a radius of convergence of

0

mean?

The series only equals the function at the single point $a$ and diverges for every other

x

; the "approximation" is useless anywhere you'd actually want to move.

What if

f^{''} (a) = 0

— does the 2nd-order term disappear, and is that a problem?

The quadratic term vanishes, so the local model is just a line (flat curvature at

a

); for Newton's method this means

H

is singular and

H^{- 1}

doesn't exist, so the pure Newton step is undefined and needs a fallback.

x = a

exactly, what step does Newton's method compute at a stationary point where

\nabla L = 0

Δ = - H^{- 1} \nabla L = - H^{- 1} 0 = 0

— it stays put, correctly signalling it has reached a critical point (though whether it is a minimum depends on

H

's sign).

Recall One-line survival guide

Divide by $n!$ ; the series is local; convergence $\neq =$ equality; expand near where you stand; gradient descent = 1st order, Newton = 2nd order with $H^{- 1}$ .

Connections

Taylor series approximation — parent note with the full derivation.
Derivatives and gradients — every coefficient is a derivative.
Gradient Descent — the 1st-order trap-free reading.
Newton's Method — the 2nd-order reading with $H^{- 1}$ .
Hessian matrix — the curvature term that can be singular.
Convex functions — when $H ⪰ 0$ the bowl opens up.
Approximation error and Big-O — the $(x - a)^{k + 1}$ remainder.