The Jacobian matrix: Foundations — AI-ML deep dive

Why this page exists

The parent note throws a lot of notation at you very fast: $R^{n} \to R^{m}$ , $\partial f_{i} / \partial x_{j}$ , $\nabla f_{i}$ , $J h$ , $det J$ , $diag (\dots)$ . If any one of those symbols is a fog to you, the whole topic collapses. So below we take every symbol the parent uses and give it three things: a plain-words meaning, a picture, and the reason the topic needs it. Read top to bottom — each block is a rung you stand on to reach the next.

1. A number line, and what $R$ means

The picture: a horizontal line stretching left and right forever, with $0$ in the middle. A single real number is one dot sitting on that line.

Why the topic needs it: every input and every output in the Jacobian story is built out of real numbers. Before we can stack numbers into vectors we must name the pool they come from.

2. Vectors and the spaces $R^{n}$

$R^{1} = R$ : one number, one dot on a line.
$R^{2}$ : two numbers $(x, y)$ , a point (or arrow) on a flat sheet.
$R^{3}$ : three numbers, a point in the room you are sitting in.

We write such a list as a vector and put it in bold: $x$ . Its individual numbers are the components $x_{1}, x_{2}, \dots, x_{n}$ — same letter, subscript for the slot.

The picture (figure above): a vector is an arrow from the origin to a point. In $R^{2}$ the arrow $x = (3, 2)$ points 3 right and 2 up. The word "vector" and the word "point" describe the same list of numbers — one emphasises the arrow, one emphasises the tip.

Why the topic needs it: the Jacobian describes machines that take a vector in and give a vector out. Without the idea of $R^{n}$ there is nothing to be multi-dimensional about.

3. A function between vector spaces: $f : R^{n} \to R^{m}$

The output is itself a list, so $f$ is really $m$ ordinary scalar functions bundled together:

f (x) = f_{1} (x_{1}, \dots, x_{n}) ⋮ f_{m} (x_{1}, \dots, x_{n}) .

Each $f_{i}$ is one output component — a single dial-reading of the output — and it is allowed to depend on all the inputs.

The picture (figure above): a box with $n$ input dials on the left and $m$ output bulbs on the right. Every dial may wire to every bulb. That web of wires is exactly what the Jacobian will measure.

Why the topic needs it: this is the object the Jacobian is about. The whole note answers "how does this box respond when I nudge the dials?"

4. Slope of one variable: the ordinary derivative $f^{'} (x)$

The picture: zoom into the curve $y = f (x)$ until it looks like a straight line. Its steepness is $f^{'} (x)$ . A big $f^{'}$ means a steep climb; a negative $f^{'}$ means going downhill; $f^{'} = 0$ means flat.

Why we use a slope at all (WHY this tool): the derivative answers the question "if I nudge $x$ by a tiny amount $h$ , how much does $f$ change?" The answer is $f^{'} (x) h$ . This single fact — change $\approx$ slope $\times$ nudge — is the seed the entire Jacobian grows from. We pick the derivative and not, say, the average value, because we care about local response to a small push, which is precisely what a slope encodes.

5. Many inputs, one output: partial derivatives $\frac{\partial f _{i}}{\partial x _{j}}$

Now let one output $f_{i}$ depend on several inputs. If I want its slope, I face a puzzle: slope in which direction? Wiggling $x_{1}$ is a different push than wiggling $x_{2}$ .

The picture (figure above): the graph of $f_{i} (x, y)$ is a landscape — a hilly surface over the flat $(x, y)$ floor. $\partial f_{i} / \partial x$ is the slope you feel walking due east (only $x$ changes). $\partial f_{i} / \partial y$ is the slope walking due north. Same spot, two different steepnesses, because a hill can be flat one way and steep another.

Why the topic needs it: every single entry of the Jacobian is one partial derivative — output $i$ , nudged by input $j$ . Partials are the atoms the table is built from.

6. Bundling the partials of one output: the gradient $\nabla f_{i}$

For output $f_{i}$ there are $n$ partials — one per input. Collect them into a single list.

The picture: back on the hill of §5, the gradient is an arrow drawn on the flat floor pointing in the steepest-uphill direction, and its length is how steep that is. It packs "east-slope" and "north-slope" into one arrow.

Why the topic needs it: the parent says "row $i$ of the Jacobian is $(\nabla f_{i})^{⊤}$ ." So one gradient, tipped on its side, becomes one row of the table. Stack $m$ of them and you have the whole Jacobian. See the deeper note Gradient and directional derivative for the "steepest ascent" story.

7. Weighted sums as a dot product

When we nudge all inputs at once by a small step $h = (h_{1}, \dots, h_{n})$ , the change in $f_{i}$ is the sum of each partial times its own nudge:

Δ f_{i} \approx \frac{\partial f _{i}}{\partial x _{1}} h_{1} + \dots + \frac{\partial f _{i}}{\partial x _{n}} h_{n} .

So the messy sum above is exactly the tidy dot product $(\nabla f_{i})^{⊤} h$ — the row-gradient of output $i$ dotted with the nudge.

Why the topic needs it: this is the single algebra move (Step 2 of the parent's derivation) that turns "a weighted sum of nudges" into "a row times a vector." Recognising it is what lets $m$ separate sums collapse into one clean matrix equation.

8. Stacking rows into a matrix, and multiplying by a vector

J (m \times n) h (n \times 1) = change in outputs (m \times 1) .

The picture: the inner numbers $n$ and $n$ touch and cancel; the outer numbers $m$ and $1$ survive as the shape of the answer. If the inner numbers don't match, the multiplication is illegal.

Why the topic needs it: the master formula $f (x + h) \approx f (x) + J h$ is a matrix–vector product. It also powers the composition rule that becomes Backpropagation and Jacobian–vector products.

Gradients and their "steepest direction" meaning → Gradient and directional derivative.
Multiplying Jacobians in the right order → The chain rule (multivariable) and Backpropagation.
The linear-map view $f (x + h) \approx f (x) + J h$ → Linear approximation and Taylor series.
Second-order slopes (a matrix of partials of a gradient) → Hessian matrix.
$∣ det J ∣$ as an area factor → Change of variables in integration.
Cheap " $J$ times a direction" without forming $J$ → Jacobian-vector products (forward-mode autodiff).

Equipment checklist

Test yourself — cover the right side and see if you can answer each before revealing.

What does

R^{n}

mean in plain words?

An ordered list of

n

real numbers — a point/arrow in

n

-dimensional space.

Why do we write some letters in bold like

x

Bold marks a whole vector (a list); plain marks a single number.

What does

f : R^{n} \to R^{m}

describe?

A machine taking a list of

n

numbers and returning a list of

m

numbers.

What is

f^{'} (x)

in one phrase?

The slope of the graph — how much

f

changes per tiny nudge in

x

What does the curly

\partial

\partial f_{i} / \partial x_{j}

promise?

All variables except

x_{j}

are held frozen while we take the slope.

What is

\nabla f_{i}

The column vector of all partial derivatives of the single output

f_{i}

What does the transpose

^{⊤}

do to

\nabla f_{i}

Tips the column into a row,

(\nabla f_{i})^{⊤}

— which becomes one row of the Jacobian.

How do you compute

J h

Dot each row of

J

with

h

; the results stack into the output vector.

Why must the inner dimensions match in

(m \times n) (n \times 1)

You dot rows of length

n

with a vector of length

n

; mismatched lengths make the dot product undefined.

When is

det J

even defined, and what does

∣ det J ∣

mean?

Only when

J

is square (

m = n

); it is the local area/volume scaling factor of the map.

What does

diag (d_{1}, \dots, d_{m})

look like?

A square matrix with

d_{i}

on the diagonal and zeros everywhere else.

Foundations — The Jacobian matrix

Why this page exists

1. A number line, and what $R$ means

2. Vectors and the spaces $R^{n}$

3. A function between vector spaces: $f : R^{n} \to R^{m}$

4. Slope of one variable: the ordinary derivative $f^{'} (x)$

5. Many inputs, one output: partial derivatives $\frac{\partial f _{i}}{\partial x _{j}}$

6. Bundling the partials of one output: the gradient $\nabla f_{i}$

7. Weighted sums as a dot product

8. Stacking rows into a matrix, and multiplying by a vector

9. Two square-matrix extras: $diag$ and $det$

10. The prerequisite map

11. Where each piece reappears later

Equipment checklist