Foundations — Data augmentation for images

This page assumes nothing. Before you can read the parent note Data augmentation for images, you need to know what every letter and squiggle in it means. We build each one from a picture, then say why the topic needs it.

0. What is an image, really?

Before any formula, we must agree on what an "image" is to a computer.

Look at the figure. On the left, a picture your eye reads as a shape. On the right, the exact same thing the computer sees: a grid of numbers. The whole topic is about doing arithmetic on this grid. When we "rotate a cat," we are really shuffling these numbers into new grid positions.

• A number like 0 = black, 255 = brightest. This range $[0,255]$ is just "how many brightness levels a byte can hold."
• The grid has a height and a width — how many rows and columns of pixels.

1. The size symbols: $H$, $W$, $C$

The picture: a colour photo is really three stacked grids — one for Red, one for Green, one for Blue. Here $C=3$. A grey photo has $C=1$. Stack the three grids and you get a box of numbers that is $H$ tall, $W$ wide, and $C$ deep.

Why the topic needs it: the parent note writes a transformation as $t_i:{\mathbb{R}}^{H\times W\times C}\to {\mathbb{R}}^{H^{\prime }\times W^{\prime }\times C}.$ We now decode this piece by piece below — but you already know $H\times W\times C$ just means "a box of numbers this big."

2. The set symbols: $\mathbb{R}$, $\times$, $\{\}$, $\in$, $\to$

These come from the language of maths, not from images — but each has a plain meaning.

3. Coordinates: $(x,y)$, their domain, primes $x^{\prime }$, and the center $(c_x,c_y)$

Why the topic needs it: every geometric augmentation is just a rule that says "the pixel at $(x,y)$ moves to $(x^{\prime },y^{\prime })$." Because addresses run $0\dots W-1$, the horizontal flip that mirrors left-to-right must send column $0$ to column $W-1$, column $1$ to $W-2$, and so on. The rule that does this is:

Check it: with zero-based indices the smallest address is $0$ and the largest is $W-1$; plugging $x=0$ gives $x^{\prime }=W-1$ (leftmost becomes rightmost) — exactly a mirror. The $-1$ is entirely because we count from 0. The vertical flip is the same idea turned on its side, mirroring top-to-bottom.

The center $(c_x,c_y)$

4. Angle $\theta$, its units, and the full rotation formula

Why $\sin$ and $\cos$, not something else? Because rotation is circular motion, and $\cos \theta$, $\sin \theta$ are exactly the horizontal and vertical positions of a point on a circle after turning by $\theta$. They are the only tools that convert "spin by an angle" into ordinary multiply-and-add.

• $\cos \theta$ = how much of the turn stays in the horizontal direction.
• $\sin \theta$ = how much leaks into the vertical direction.
• At $\theta =0$: $\cos 0=1$, $\sin 0=0$ → the point doesn't move (correct — no rotation).
• At $\theta =\frac{\pi }{2}$ (a quarter-turn): $\cos =0$, $\sin =1$ → right becomes up.

That is exactly the matrix formula in the parent note, written out number-by-number so you can hand-compute it. Why subtract $c_x,c_y$ first? Rotation formulas only work around the origin $(0,0)$; subtracting the center moves the pin to the origin, and adding it back afterwards slides everything home.

Landing between pixels: interpolation

5. Probability words: $P$, $\mathbb{E}$, $\sim$, $\lambda$, distributions

The parent's "Mathematical Framework" uses probability. Here is the minimum.

The gap between $P_{\text{train}}$ (few photos) and $P_{\text{test}}$ (the real world) is exactly overfitting — see 5.2.3-Overfitting-vs-underfitting. Augmentation shrinks that gap by faking extra samples.

6. Loss $\ell$, the model $f_{\theta }$, and $\theta$ (again!)

Why the topic needs it: augmentation changes the input $x$ into $t(x)$ but keeps the label $y$. So training minimizes $\ell (f_{\theta }(t(x)),y)$ — "be right even on the wobbled version." This is what drives the model toward invariances and better generalization.

7. How these foundations feed the topic

Each box on the left is a prerequisite; follow the arrows and you arrive at the parent topic. Related regularizers that share this goal include 3.4.8-Dropout-in-neural-networks and, for reusing pretrained features, 3.4.12-Transfer-learning-CNs.

Equipment checklist

Test yourself — reveal only after you answer aloud.