Intuition The one idea behind everything hereA hash is a tiny fingerprint computed from a file's contents ; if you store heavy files by that fingerprint and keep only the fingerprint under Git, you get all of Git's version-control powers on data thousands of times too big for Git to hold. That single trick — "point at the data by its fingerprint" — is the whole of DVC.
This page assumes you have read nothing about DVC. Before you can follow the parent note Data versioning (DVC) , you need seven building blocks. We build each one from zero: plain words → the picture → why the topic needs it . Each block leans on the one before it.
Definition A file = an ordered list of bytesA byte is a number from 0 255 file is just a long row of these boxes, in a fixed order. A 2 GB file is a row of about two billion of these boxes.
Look at the top row in the figure: the file is drawn as a strip of little numbered cells. That strip — nothing more — is "the data".
Intuition Why the topic needs thisThe parent note keeps saying "2 GB dataset", "one byte changed", "the actual bytes". All of that only makes sense once you see a file as a fixed-order row of bytes . Everything DVC does is about moving and fingerprinting this row.
Notice two files can hold the same row of bytes even if they have different names — the name is a label written outside the strip, not part of it. Hold onto that; Block 3 depends on it.
Definition A hash functionA hash function takes any row of bytes and spits out one short fixed-length string, like a1b2c3.... We write it as
h = MD5 ( bytes of file ) h h MD5 is one particular such machine (there are others like SHA-256); the parent note happens to use MD5.
The middle row of the figure shows the byte-strip going into a machine box labelled MD5 and a short code h
Three properties make a hash useful — each shown as a separate mini-panel in the next figure:
Deterministic (left panel): same bytes in → always the same h Avalanche (middle panel): change one byte, and h Fixed, tiny size (right panel): a 2 GB file and a 3-byte file both produce the same length short code. The fingerprint never grows with the data.
Intuition Why "opposite over adjacent"-level care matters hereThese three properties are the entire reason DVC works. Deterministic ⇒ the same data is recognised again. Avalanche ⇒ any edit is detected automatically. Tiny fixed size ⇒ the fingerprint is small enough to live inside Git. If you only remember one thing: h
Common mistake "Different files might get the same hash by accident."Why it feels right: there are infinitely many files but only finitely many 32-character codes, so collisions must exist in theory . Why it's practically wrong: the number of possible MD5 codes is astronomically large (1 6 32 h What it looks like: two different byte-strips in the figure produce visibly different h
Definition Content-addressable storageNormally you find a file by its name/path ("data/train.csv"). Content addressing flips this: you find a file by its content hash h stored under its own fingerprint as its filename.
In the figure, watch the amber arrow: the byte-strip is filed into a drawer labelled with its hash a1/b2c3..., not with the human name.
Two payoffs fall out for free:
Deduplication. If two people add the exact same dataset, both compute the same h once . The drawer already exists.Automatic version detection. Edit one byte → avalanche → new h the old version still exists . That is versioning, for free.
Intuition Why the topic needs thisThis is the deep idea the parent note points at with Content-addressable storage . DVC's cache .dvc/cache/a1/b2c3... is literally a content-addressed drawer. Once you see "filed by fingerprint, not name", the DVC cache stops being mysterious.
A pointer is a tiny text file that does not contain the data — it contains the fingerprint of the data plus a bit of bookkeeping. DVC's pointer is data.csv.dvc:
outs : - md5 : a1b2c3... size : 2147483648 path : data.csv Read it as: "there should be a file named data.csv, it is 2147483648 bytes (≈ 2 a1b2c3...."
In figure s03, the small card next to the drawer is the pointer: human name on the front, fingerprint written inside telling you which drawer to open.
Intuition Why this is the whole trickThe pointer is ~100 bytes — small enough for Git . The 2 GB strip stays in the drawer. So Git version-controls the pointer , and the pointer version-controls the data (via h
Recall Check yourselfIf two commits contain pointers with the same md5, do they refer to the same data? ::: Yes — same h
Definition Git, in one lineGit version control is a system that records snapshots of your files over time; each snapshot is a commit , and you can jump back to any past commit with git checkout. Git itself already fingerprints file contents internally — so it, too, is content-addressable.
Picture a shelf of numbered snapshots; checkout slides you to any one of them and your working folder becomes exactly what it was then.
Common mistake "Then why not put the 2 GB file straight into Git?"Why it feels right: Git snapshots files, and data is a file. Why it's wrong: Git keeps a full copy of every version inside .git/. Ten edits to a 5 GB file ≈ 50 GB repo; cloning crawls, and Git can't meaningfully diff binary blobs. What it looks like: the shelf overflowing with near-identical giant boxes. Fix: put only the ~100-byte pointer on the Git shelf — that's Block 4.
Intuition Why the topic needs thisDVC does not replace Git — it rides on it . Git versions the small pointer; the pointer's h git checkout do before "checkout the pointer, then checkout the data" makes sense.
Definition Remote object storageA remote is a big cheap warehouse for bytes, sitting on the network — e.g. Object storage (S3/GCS) like s3://mybucket/.... It stores objects by key (a name/fingerprint), which is exactly content addressing again, on someone else's disks.
Picture your laptop's local cache as a small drawer and a giant garage across town holding copies. dvc push = truck the drawers to the garage; dvc pull = truck them back.
Intuition Why the topic needs thisThe pointer resolves to "data at hash h exist somewhere your teammate can reach . Git commits carry only the pointer; the bytes live in cache/remote. This is why the parent note stresses: commit ≠ shipped data ; you must dvc push.
Definition ReproducibilityAn experiment is reproducible if you can restore, together, the exact code , the exact data , and the exact outputs that produced a result — see Reproducibility in ML .
Definition DAG (directed acyclic graph)A DAG is a set of boxes joined by one-way arrows with no loops — here: data → train.py → model.pkl. "Directed" = arrows have a direction; "acyclic" = you can never follow arrows back to where you started. A DVC pipeline (dvc.yaml) is such a DAG.
The figure draws it: a Git commit (bottom) contains the code hash and the data pointer's h resolves to the byte-strip in the drawer. One commit pins both — that binding is the parent note's "versioning invariant":
Git commit contains .dvc pointer ( h ) resolves to data at hash h
Intuition Why the topic needs thisThis is the payoff of Blocks 1–6. Because h as a unit . That unity feeds MLOps pipelines and pairs with Experiment tracking (MLflow) .
Bytes: a file is an ordered row of bytes
Hash: tiny fixed fingerprint of the bytes
Content addressing: store by fingerprint
Pointer file: tiny note holding the hash
Git: versions small files snapshots
Remote storage: cheap warehouse for bytes
Reproducibility and DAG: bind code plus data
DVC: version large data with Git-sized pointers
Test yourself — reveal only after answering.
A file, stripped of its name, is fundamentally what? An ordered row of bytes (numbers 0–255).
What does a hash function output, and does its length depend on file size? A short fixed-length code h
Deterministic property of a hash means? The same bytes always produce the same h
Avalanche property means? Changing one byte changes h
Content addressing stores/finds a file by its ___ instead of its ___? By its content hash instead of its name/path.
Two payoffs of content addressing? Deduplication (same content stored once) and automatic version detection (new content → new hash).
What does a .dvc pointer file contain, and roughly how big is it? The data's md5 hash, size, and path — about 100 bytes.
What does Git store for every version of a file, and why is that bad for big data? A full copy of each version in .git/; huge repos and slow clones.
A commit only carries the pointer — where do the actual bytes live until you share them? In your local cache, until dvc push sends them to the remote.
What is a remote in DVC terms? Cheap networked object storage (e.g. S3/GCS) keyed by hash/name.
A DAG is? Boxes joined by one-way arrows with no loops (directed, acyclic).
Reproducibility binds which three things? Exact code, exact data, exact outputs.
Data versioning (DVC) — the parent topic these foundations unlock.Git version control — versions the small pointer; DVC reuses its content-hashing idea.Content-addressable storage — Block 3, the hashing principle under Git, DVC and Docker.Object storage (S3/GCS) — Block 6, the remote warehouse for heavy bytes.Reproducibility in ML — Block 7, why binding code + data matters.MLOps pipelines — where the dvc.yaml DAG is put to work.Experiment tracking (MLflow) — pairs with DVC on the reproducibility problem.5.3.04 Data versioning (DVC) (Hinglish) — same material in Hinglish. 
