Foundations — Taylor series approximation

This page assumes nothing. Before you can read the parent note Taylor series approximation, you must own every symbol it throws at you. We build each one from a picture, in an order where each tool leans on the previous.

1. The function $f$ and the point $a$

The letter $f$ is just a name for the machine; $f(x)$ (read "$f$ of $x$") means "the output when the input is $x$". The parenthesis is not multiplication — it's "feed this in".

Notice the quantity $(x-a)$ in that figure — the pale-yellow gap. It answers the question "how far did I walk away from the flag?" Taylor is written entirely in powers of this gap, because when you stand at the flag ($x=a$) the gap is $0$, and $0$ raised to any positive power vanishes. That vanishing is the trick that lets us peel off one coefficient at a time.

2. Slope — the derivative $f^{\prime }(a)$

Before curvature or series, you need slope: how steep is the curve, and which way does it tilt?

A curve doesn't have one slope — it bends. So we ask for the slope exactly at the flag, by drawing the line that just kisses the curve there.

The little tick mark in $f^{\prime }$ means "differentiate once". Two ticks $f^{\prime \prime }$ means "do it twice", and so on. When ticks get clumsy we write $f^{(n)}$ for "the $n$-th derivative".

3. Curvature — the second derivative $f^{\prime \prime }(a)$

Slope tells direction. But two curves can share a slope and still peel apart — one bends up, one bends down. We need bend.

4. Powers and the factorial $n!$

Where does this factorial come from? Watch what differentiating a pure power does:

$\frac{d}{dx}(x-a{)}^n=n(x-a{)}^{n-1}$

Each time you differentiate, the exponent falls out front as a multiplier. Do it $n$ times to $(x-a{)}^n$ and you collect $n\times (n-1)\times \cdots \times 1=n!$, and the power drops all the way to $(x-a{)}^0=1$.

5. The summation sign $\sum$

The parent writes the whole series as one compact symbol. Decode it.

${\sum }_{n=0}^3(x-a{)}^n=(x-a{)}^0+(x-a{)}^1+(x-a{)}^2+(x-a{)}^3$

Putting $\infty$ on top means "keep going forever" — that's a series (an infinite sum). Stopping at a finite $k$ gives a polynomial (a finite sum of powers), which is what a computer can actually evaluate.

6. The gradient $\nabla$ and Hessian $H$ (many inputs)

ML losses depend on many numbers at once (all the weights). The single-input tools upgrade:

The symbol ${\nabla }^{\top }$ (a small "T") means we lay that arrow flat as a row so it can multiply the step vector $\Delta$ — this row-times-column is a dot product, which just totals up "slope in each direction $\times$ distance moved in that direction". See Derivatives and gradients.

Prerequisite map

Equipment checklist

Test yourself — reveal only after you have answered aloud.