Exercises — Taylor series approximation

WHAT: Read off the $x^3$ term: its coefficient is $\frac{1}{6}$, so $c_3=\frac{1}{6}$. WHY: Every Taylor coefficient obeys $c_n=\frac{f^{(n)}(a)}{n!}$. Here $a=0$ and $n=3$, so $c_3=\frac{f^{\prime \prime \prime }(0)}{3!}=\frac{f^{\prime \prime \prime }(0)}{6}$. Setting $\frac{f^{\prime \prime \prime }(0)}{6}=\frac{1}{6}$ gives $f^{\prime \prime \prime }(0)=1$. Answer: $c_3=\frac{1}{6}$, and $f^{\prime \prime \prime }(0)=1$.

A Maclaurin series is defined as a Taylor series with anchor point $a=0$. Only (b) has $a=0$. Answer: (b). (a) is anchored at $1$; (c) at $2$ — both are Taylor but not Maclaurin.

$f(x)=\cos x\Rightarrow f(0)=\cos 0=1$. $f^{\prime }(x)=-\sin x\Rightarrow f^{\prime }(0)=-\sin 0=0$. $f^{\prime \prime }(x)=-\cos x\Rightarrow f^{\prime \prime }(0)=-\cos 0=-1$. Answer: $1,0,-1$. (These are the "value, velocity, vroom" of $\cos$ at zero.)

Step 1 — value. $f(4)=\sqrt{4}=2$. Step 2 — slope. $f^{\prime }(x)=\frac{1}{2}{x}^{-1/2}$, so $f^{\prime }(4)=\frac{1}{2}\cdot \frac{1}{2}=\frac{1}{4}$. Step 3 — curvature. $f^{\prime \prime }(x)=-\frac{1}{4}{x}^{-3/2}$, so $f^{\prime \prime }(4)=-\frac{1}{4}\cdot \frac{1}{8}=-\frac{1}{32}$. Assemble using $P_2(x)=f(a)+f^{\prime }(a)(x-a)+\frac{f^{\prime \prime }(a)}{2}(x-a{)}^2$: $P_2(x)=2+\frac{1}{4}(x-4)-\frac{1}{64}(x-4{)}^2.$ Check: at $x=4.4$, $P_2=2+0.1-\frac{0.16}{64}=2.0975$, and $\sqrt{4.4}=2.0976$ — excellent.

$e^x=1+x+\frac{x^2}{2}+\frac{x^3}{6}+\dots$ At $x=0.5$: $1+0.5+\frac{0.25}{2}+\frac{0.125}{6}=1+0.5+0.125+0.0208\overline{3}=1.6458\overline{3}$. Answer: $\approx 1.64583$. True value $e^{0.5}=1.64872$, so we're off by about $0.003$.

From the parent note, $\sin x=x-\frac{x^3}{6}+\dots$, so $P_3(x)=x-\frac{x^3}{6}$. At $x=0.2$: $0.2-\frac{0.008}{6}=0.2-0.0013\overline{3}=0.198\overline{6}$. Answer: $\approx 0.198667$. True $\sin 0.2=0.198669$ — accurate to 5 decimals.

At $x=0.5$: $1+0.5+0.25+0.125=1.875$. The true value $\frac{1}{1-0.5}=2$. The partial sum is climbing toward $2$ — adding more terms keeps improving it. At $x=2$: $1+2+4+8=15$, while the true value $\frac{1}{1-2}=-1$. The partial sum is running away to $+\infty$, nowhere near $-1$. What it reveals: convergence is local. Inside the radius $|x|<1$ the series homes in on the function; outside it, the terms grow and the sum diverges even though the function itself is perfectly finite. See Approximation error and Big-O.

WHAT the figure shows: the black curve is $e^x$; the coral line is $P_1(x)=1+x$; the mint curve is $P_2(x)=1+x+\frac{x^2}{2}$. WHY $P_2$ is better: Taylor's remainder says the leftover error of an order-$k$ polynomial shrinks like $(x-a{)}^{k+1}$. For $P_1$ ($k=1$) the error scales like $(x-a{)}^2$; for $P_2$ ($k=2$) it scales like $(x-a{)}^3$. When $|x-a|<1$, a cube is much smaller than a square, so $P_2$'s error is dramatically smaller. WHAT IT LOOKS LIKE: the coral line peels away from the curve as soon as $x$ bends; the mint curve, having matched the curvature too, follows the bend and stays glued far longer.

The derivatives cycle every 4 steps: $\sin ,\cos ,-\sin ,-\cos ,\sin ,\dots$ At $x=0$: $\sin 0=0$, $\cos 0=1$, $-\sin 0=0$, $-\cos 0=-1$, then repeat. So the even-order derivatives ($f,f^{\prime \prime },f^{(4)},\dots$) are $0,0,0,\dots$ (all the $\pm \sin 0$ ones), and the odd-order derivatives are $1,-1,1,\dots$ Since the coefficient $c_n=\frac{f^{(n)}(0)}{n!}$, every even $c_n$ is $0$ — those terms vanish — leaving only odd powers $x,x^3,x^5,\dots$ This mirrors the fact that $\sin$ is an odd function ($\sin (-x)=-\sin x$): only odd powers respect that symmetry.

$L(\theta )={\theta }^2$, $L^{\prime }(\theta )=2\theta$, $L^{\prime \prime }(\theta )=2$. At ${\theta }_0=3$: $L=9$, $L^{\prime }=6$, $L^{\prime \prime }=2$. Expansion (which is exact here because $L$ is already degree 2, so all higher derivatives are 0): $L(3+\Delta )=9+6\Delta +\frac{1}{2}\cdot 2\cdot {\Delta }^2=9+6\Delta +{\Delta }^2.$ Newton step $\Delta =-\frac{L^{\prime }({\theta }_0)}{L^{\prime \prime }({\theta }_0)}=-\frac{6}{2}=-3$. New point: ${\theta }_0+\Delta =3-3=0$, and $L(0)=0$ is the true minimum. Newton reaches it in one step because a quadratic is its own 2nd-order Taylor model — the model has no error to correct. See Newton's Method.

$L^{\prime }(3)=6$, so new $\theta =3-0.1\cdot 6=3-0.6=2.4$. Why not the minimum: gradient descent uses only the 1st-order Taylor picture, $L({\theta }_0+\Delta )\approx L({\theta }_0)+L^{\prime }({\theta }_0)\Delta$. This straight-line model has no bottom of its own — it just says "downhill is negative $\Delta$" — so we must guess a step size $\eta$ and creep. Newton adds the curvature term, which does have a bottom, and jumps straight to it. See Gradient Descent.

Gradient: $\nabla f=(2x,6y)$, so at $(1,1)$, $\nabla f=(2,6)$. Value: $f(1,1)=1+3=4$. Terms with $\Delta =(0.1,-0.1)$:

• Gradient term: $\nabla f^{\top }\Delta =2(0.1)+6(-0.1)=0.2-0.6=-0.4$.
• Curvature term: $\frac{1}{2}{\Delta }^{\top }H\Delta =\frac{1}{2}(2(0.1{)}^2+6(-0.1{)}^2)=\frac{1}{2}(0.02+0.06)=0.04$. Taylor estimate: $4-0.4+0.04=3.64$. Exact: $f(1.1,0.9)=1.21+3(0.81)=1.21+2.43=3.64$. They match exactly — again because $f$ is quadratic, so the 2nd-order Taylor model is perfect (no cubic-and-up terms exist). The Hessian matrix $H$ carries all the curvature.

Maclaurin series predicts $g(0.5)\approx 0$ (every term is zero). True value: $g(0.5)=e^{-1/(0.5{)}^2}=e^{-1/0.25}=e^{-4}\approx 0.018316$. What it proves: a function can be perfectly smooth (infinitely differentiable) yet its Taylor series fails to equal it at any nonzero point. Matching all derivatives at one spot does not guarantee reconstruction. This is the sharpest warning that Taylor is a local, conditional tool — echoing the parent note's first mistake callout.

$f^{(4)}(x)=e^x$, so $f^{(4)}(\xi )=e^{\xi }$ with $0<\xi <0.5$. Since $e^x$ increases, the largest possible value is $e^{0.5}<e^1=2.718\dots$, but a safe simple bound is $e^{0.5}<2$ (since $e^{0.5}\approx 1.6487$). $|R_3|=\frac{e^{\xi }}{4!}(0.5{)}^4<\frac{2}{24}\cdot 0.0625=\frac{0.125}{24}=0.0052083\dots$ Bound: $|R_3|<0.00521$. Check against reality: the actual error from L2.2 was $|1.64872-1.64583|=0.00289$, which indeed sits under our bound $0.00521$. ✓ The bound is honest and not violated.

Write the exact order-1 Taylor with remainder: $f(x)=\underset{\text{tangent line at }a}{\underbrace{f(a)+f^{\prime }(a)(x-a)}}+\underset{R_1}{\underbrace{\frac{1}{2}{f}^{\prime \prime }(\xi )(x-a{)}^2}}.$ The tangent-line part is exactly the straight line touching $f$ at $a$. The remainder $R_1=\frac{1}{2}{f}^{\prime \prime }(\xi )(x-a{)}^2$ has $(x-a{)}^2\ge 0$ always. If $f^{\prime \prime }(\xi )\ge 0$ everywhere, then $R_1\ge 0$. Therefore $f(x)\ge \text{tangent line}$ for every $x$ — the graph never falls below any tangent. That is precisely the definition of convexity. See Convex functions. The Hessian being positive semi-definite ($H⪰0$) is the multivariable version of "$f^{\prime \prime }\ge 0$".

If you can (a) build $P_2$ for any $f$ at any $a$, (b) explain why the error scales like $(x-a{)}^{k+1}$, (c) derive Newton's step $-H^{-1}\nabla L$ from a 2nd-order expansion, and (d) name why $e^{-1/x^2}$ breaks the "series = function" myth — you've mastered this node. Return to Taylor series approximation to see the map.