Foundations — Experiment tracking and reproducibility

This page assumes you have seen nothing. Every symbol used in the parent note is built here, one at a time, each resting on the one before it. Read top to bottom.

0. What a "function" even means here

Before any symbol, the single mental model.

We will write this machine as the letter $f$. The letter is just a name for "the training process". Nothing magic.

Look at the figure: five arrows go into the box $f$, two arrows come out. That entire picture is the whole topic. Everything below is naming those seven arrows.

1. The symbol $f$ — the training run

How to read this line, left to right, out loud:

• The stuff in the round brackets after $f$ — that's what we put in (the five ingredients).
• The stuff on the far left before the $=$ — that's what comes out (the two products).
• The $=$ means "the left side is produced by the right side".

Why the topic needs $f$: if you cannot see a training run as a function, you cannot ask "which input caused the output to change?" — and that question is experiment tracking.

2. The five inputs (the ingredients)

Each input is one arrow going into the box. We build them in the order a real run uses them.

3. Notation you must not fear: the git commit SHA

Why a fingerprint and not a name like "version 2"? Names get reused and lie ("final_v2_REAL_final"). A SHA is computed from the content itself, so different content is guaranteed a different SHA. That guarantee is exactly what we exploit next for data.

4. The tool "hash" — and WHY this tool

Here a mathematical tool enters, so we justify it explicitly.

Why a hash and not "just look at the file"? A dataset can be millions of rows — you cannot eyeball whether it changed. You need a tool that answers one yes/no question cheaply: "is this the exact same file as before?" A hash answers exactly that. Comparing two short fingerprints is instant; comparing two giant files row-by-row is not.

In the figure: two files that differ by a single cell produce completely different fingerprints (red). That sensitivity is the whole point — a silent data edit cannot hide.

5. The two outputs (the products)

6. New symbols for the math: $p_i$, $n$, $\prod$, $ϵ$

The parent note uses a product formula. Every symbol in it, from zero.

Why multiply, and why THIS tool? We want the chance that all stages behave the same at once. When independent things must all happen together, their chances multiply — like flipping four coins and asking "all heads?" ($\frac{1}{2}\times \frac{1}{2}\times \frac{1}{2}\times \frac{1}{2}$). Addition would answer "at least one" — the wrong question. So multiplication is chosen precisely because "reproducible" means "every stage reproducible simultaneously".

The figure shows the parent's worked pipeline: $1.0\times 0.7\times 0.9\times 1.0=0.63$. Notice the weakest bar (0.7, red) drags the whole product down — one leaky stage caps everything, no matter how perfect the others are.

7. Putting it together: the run tuple

Now every symbol above lets you read the parent's central object without fear.

A tuple is just an ordered bundle of fields — a labelled row. run_id is the unique name of this run (its own SHA-like fingerprint) so you can point to it later. Every other field you already met in §2 and §5. Nothing here is decorative: the five input fields are the five arrows into $f$, the two output fields are the two arrows out.

Prerequisite map

Every foundation box on the left flows into the topic box at the bottom. If any left box is fuzzy, the topic will feel like magic instead of engineering.

Equipment checklist

Test yourself — read the prompt, answer aloud, then reveal.

See also: CI-CD for Machine Learning automates re-running $f$ on every change, and 5.3.02 Experiment tracking and reproducibility (Hinglish) for the same ideas in Hinglish. Return to the parent: Experiment tracking and reproducibility (index 5.3.2).