4.8.29 · D1Numerical Methods

Foundations — Solving nonlinear systems — Newton's method in n dimensions

2,153 words10 min readBack to topic

This page assumes nothing. Before you can read the parent topic, you need to recognise every squiggle it writes. Below, each symbol gets three things: what it means in plain words, the picture it stands for, and why the topic can't proceed without it. They are ordered so each one leans only on the ones above it.


1. A number line, a plane, and

Picture. Look at figure s01: a single point sitting in the plane, described by its two coordinates.

Figure — Solving nonlinear systems — Newton's method in n dimensions

Why the topic needs it. The whole problem is "find the special point in where everything vanishes." If you don't know that is a point made of numbers, the boldface below looks like magic.


2. The bold letter — a vector

Picture. In figure s01 the arrow from the origin to the point is the vector . Its two numbers are how far right and how far up you walk.

Why the topic needs it. In one dimension we had one unknown . Here we have unknowns at once, and bundling them into one bold lets us write equations as a single clean line.


3. A function of several variables:

Picture. Feed it a point on the paper, get a height. If you plot that height above every point, you get a landscape — a hilly surface floating over the plane. The places where the height is exactly trace out a curve on the floor: the "sea-level line" of that landscape.

Why the topic needs it. Each equation is really "walk along the sea-level line of landscape ." Solving the system = finding the point where all the sea-level lines cross.


4. The bold and the equation

Picture. Figure s02 draws two curves — a circle () and a parabola (). Each is one landscape's sea-level line. The red dot where they cross is : the only place both are zero together.

Figure — Solving nonlinear systems — Newton's method in n dimensions

Why the topic needs it. This crossing point is the entire goal. Everything Newton does is machinery to hunt this red dot down.


5. Slope of a curved rule: the partial derivative

Before the matrix, we need one honest sub-idea: how steep is a landscape if I walk in just one direction?

Why this tool and not an ordinary derivative? An ordinary derivative needs a function of one variable. Here depends on many. So we ask the slope one direction at a time, and the curly is exactly the notation for "slope while everything else is held fixed." It's the smallest honest generalisation of .

Picture. Stand on the hill. Face east and feel the tilt under your feet — that's . Now face north and feel that tilt — that's . Two different steepnesses at the same spot. Figure s03 shows both slope arrows on one surface slice.

Figure — Solving nonlinear systems — Newton's method in n dimensions

Why the topic needs it. A curved rule has no single slope. To pretend it's a flat ramp we must know its tilt in every input direction — and each tilt is one partial derivative.


6. Collecting all slopes: the Jacobian

Picture. Think of as the "steepness report card" of the whole system at one point: one row per equation, one column per unknown. It is the flat-ramp stand-in for the bumpy landscapes — the Jacobian matrix is literally the -dimensional version of the number .

Why the topic needs it. Newton replaces curved by a ramp. A ramp in dimensions is described exactly by this grid of slopes. No Jacobian ⇒ no ramp ⇒ no method.


7. Reading a matrix acting on a vector:

Picture. says: "if I step by the little vector , the ramp predicts my outputs change by this amount." It's the flat-ramp's estimate of how moves.

Why the topic needs it. The core Newton equation is a matrix-times-vector statement. You must be able to read it before you can solve it.


8. The linear system and how we crack it

Why the topic needs it. Every Newton step is one such linear solve. The point of linearizing was precisely to reach a problem this toolbox already crushes.


9. Distance and error: and quadratic shrinking

Picture. Figure s04 plots the error shrinking : it doesn't just fall, it plummets, because each step feeds on the square of the last.

Figure — Solving nonlinear systems — Newton's method in n dimensions

Why the topic needs it. This is the payoff — the reason we tolerate building Jacobians and solving systems. Compare with plain Fixed-point iteration, which only shrinks the error by a fixed fraction each step (linear). The subscript on , simply counts iterations: is your start, after one step, and so on.


How it all feeds the method

Points as lists of numbers Rn

Vector x and function F

F equals zero means find the root

Partial derivative one slope

Jacobian J all slopes

Linear model J h equals minus F

Solving A h equals b elimination

One Newton step

Repeat and watch error square

Norm measures error size

Read it top to bottom: numbers become vectors, vectors feed ; slopes stack into the Jacobian; Jacobian plus the "hit zero" wish gives a linear model; the linear solver turns that into one step; repeat, and the norm lets us watch the error crash.


Equipment checklist

Self-test: cover the right side and see if you can answer before revealing.

What does mean?
The space of points described by real numbers each.
Is the small number in a power or an index?
An index (subscript) — it names the 2nd coordinate, not squared.
What does bold ask for?
A point where all component functions are zero at once.
What does denote?
The root — the true solution we are hunting.
In words, what is ?
The slope of when only is nudged and all other variables are held fixed.
What sits in row , column of the Jacobian ?
The partial derivative .
What does compute?
The flat-ramp's prediction of how changes when you step by .
How do we actually solve in practice?
Gaussian elimination / LU decomposition — not by forming the inverse.
When is the matrix invertible?
When its determinant is nonzero.
What does measure?
The length of the vector, .
What is ?
The error , the gap between guess and the true root.
What does quadratic convergence mean numerically?
Each error is about the square of the previous — correct digits roughly double per step.