Data versioning (DVC): Worked examples — AI-ML deep dive

Before we compute anything, let's earn the two words we lean on constantly.

The scenario matrix

Every DVC question is really one of these cells. The "axis" is: did the content change, and where do the bytes physically live?

Cell	Trigger	What DVC does	New bytes stored?
A. First add	new file, `dvc add`	hash it, move to cache, write pointer	yes (full file)
B. No change	`dvc add` on identical file	same hash → nothing new	no
C. Byte flip	1 byte edited, `dvc add`	new hash → new full copy	yes (full file)
D. Exact duplicate	two files, same content	same hash → stored once	no (dedup)
E. Empty / 0-byte file	`dvc add` empty file	hashes the empty content	tiny (still a hash)
F. Restore, cache has it	`git checkout` + `dvc checkout`	pointer → cache → workdir	no (copy from cache)
G. Restore, cache empty	fresh clone, `dvc checkout`	pointer → cache miss → needs `dvc pull`	fetched from remote
H. Pipeline skip	`dvc repro`, deps unchanged	dep hashes match → skip stage	no
I. Word problem	storage-cost estimate	count distinct hashes × size	arithmetic
J. Exam twist	rename vs re-content	content-hash blind to name	depends

The 8 worked examples below cover all ten cells (some examples hit two).

Worked examples

Worked example Example 1 — Cell A (first add) + Cell E (0-byte edge)

You run dvc add data/train.csv on a fresh 300 MB file, then dvc add notes/empty.log on a 0-byte file. What lands in Git and what lands in the cache for each?

Forecast: Guess — does the empty file get a .dvc pointer at all? Does a 0-byte file even have a hash?

Hash the content. DVC computes $h = MD5 (bytes)$ . For a 0-byte file the "bytes" are the empty string, and even that has a well-defined MD5: d41d8cd98f00b204e9800998ecf8427e.
- Why this step? Content addressing means everything gets a fingerprint, including nothing. Emptiness is still content.
Move to cache, renamed by hash. train.csv (300 MB) → .dvc/cache/<first2>/<rest>. empty.log (0 bytes) → its own hash path.
- Why this step? The cache is keyed by fingerprint, not by name, so lookup is hash → bytes.
Write the pointer. Two files appear: train.csv.dvc (size 314572800) and empty.log.dvc (size 0). Both ~100 bytes.
- Why this step? Git only ever tracks these tiny text pointers.

Verify: $300 MB = 300 \times 1024 \times 1024 = 314, 572, 800$ bytes — that's the size field written in the pointer. The empty file still produces a valid MD5, so dvc add succeeds (no crash on emptiness). ✓

Worked example Example 2 — Cell B (no change) vs Cell C (byte flip)

You have data.bin (2 GB) already added. Case (i): you run dvc add data.bin again with no edits. Case (ii): you flip exactly 1 byte then dvc add. How much NEW data enters the cache in each case?

Forecast: Same file → surely nothing new. One byte → surely only ~1 byte extra... right?

Case (i): re-hash unchanged content. The bytes are identical → MD5 is identical → the cache already contains that hash.
- Why this step? A hash is a pure function of content; unchanged content gives an unchanged fingerprint.
Result (i): 0 new bytes. DVC sees the hash present, writes the same pointer. Nothing copied.
Case (ii): re-hash the flipped file. One byte differs → MD5 is completely different (avalanche effect).
- Why this step? Hashes are designed so a tiny input change scatters the whole output — that's how change is detected reliably.
Result (ii): a FULL new 2 GB copy. DVC's default is whole-file hashing, so the new hash means a new cache object of the entire file. The old 2 GB stays too.
- Why this step? DVC versions snapshots, not diffs. It has no byte-level delta by default.

Verify: After (i) the cache holds 1 object (2 GB). After (ii) it holds 2 objects (4 GB total). Net new bytes: $0$ then $2 GB = 2147483648$ bytes. This is the parent note's "Forecast-then-Verify" made concrete. ✓

Worked example Example 3 — Cell D (exact duplicate, deduplication)

a.csv and b.csv are byte-for-byte identical, 500 MB each. You dvc add both. How many cache objects and how many total cache bytes?

Forecast: Two files, two adds — two copies? Or does content-addressing collapse them?

Hash both. Identical content → identical MD5 $h_{a} = h_{b}$ .
- Why this step? The cache key is the fingerprint; identical fingerprints collide on purpose.
Store once. The first add creates the cache object. The second add finds the hash already present → stores nothing.
- Why this step? This IS deduplication — same content is stored exactly once.
Two pointers, one object. a.csv.dvc and b.csv.dvc both carry the same md5: value.

Verify: Cache objects = 1. Total cache bytes = $500 MB = 524, 288, 000$ , not $1000$ MB. Two pointers reference one blob. ✓

Worked example Example 4 — Cells F & G (restore with cache hit vs cache miss)

Two teammates check out commit v1 (pointer hash h1). Alice has been on the repo all along; Bob just did a fresh git clone. Both run dvc checkout. Who gets their data, who gets an error, and what fixes it?

Forecast: They ran the identical commands — should the outcome differ?

Both restore the pointer. git checkout v1 puts data.csv.dvc with hash h1 in the working directory. Identical for both.
- Why this step? Git moves the ticket, not the coat.
dvc checkout looks up h1 in the local cache.
- Why this step? Checkout copies bytes from cache into the workdir; it never talks to the remote.
Alice: cache HIT. She used h1's data before, so .dvc/cache holds it → data.csv reappears. Done.
Bob: cache MISS. A fresh clone has an empty cache → dvc checkout restores nothing (the parent note's mistake #4).
Fix for Bob: dvc pull fetches h1 from the remote into his cache, then dvc checkout (or dvc pull does both).
- Why this step? pull = cache from remote; checkout = workdir from cache. Different legs of the journey.

Verify: Path for the bytes: remote --(dvc pull)--> cache --(dvc checkout)--> workdir. Bob's failure is a missing first leg, not a broken pointer. ✓

Worked example Example 5 — Cell H (pipeline skip vs re-run)

Your dvc.yaml has a train stage: deps: [train.py, data/train.csv], outs: [model.pkl]. You run dvc repro. Scenario (i): nothing touched. Scenario (ii): you edit train.py. Which runs, which skips?

Forecast: dvc repro always retrains, right? Or does it think first?

DVC hashes every dep. It computes MD5 of train.py and data/train.csv and compares to the hashes recorded in dvc.lock.
- Why this step? A stage is a pure function of its inputs; if inputs are unchanged the output can't change.
Scenario (i): all dep hashes match → SKIP. DVC prints "stage 'train' didn't change, skipping". model.pkl is untouched.
- Why this step? This is the make idea — don't redo work whose inputs are identical.
Scenario (ii): train.py hash differs → RUN. DVC executes python train.py, produces a new model.pkl, records new hashes.
- Why this step? A changed input can change the output, so the stage is invalidated and re-executed.

Verify: The decision is purely "dep-hash matches recorded hash?" → equal ⇒ skip, unequal ⇒ run. Editing any dep (including data/train.csv) flips the same switch. ✓

Worked example Example 6 — Cell I (storage-cost word problem)

A team snapshots a 4 GB dataset once a week for 12 weeks. Of those 12 snapshots, weeks 3–7 are byte-for-byte identical (someone forgot to update the data). How much does the DVC remote actually store? Compare to naive "12 copies".

Forecast: Naive = $12 \times 4 = 48$ GB. Guess how much dedup saves before computing.

Count distinct contents. 12 weekly hashes, but weeks 3,4,5,6,7 share one hash → those 5 collapse to 1 distinct blob.
- Why this step? Storage cost = (number of distinct hashes) × size, not number of snapshots.
Distinct blobs $= 12 - (5 - 1) = 8$ . (Five identical weeks become one; that removes 4 duplicates.)
- Why this step? Content-addressing stores each unique content exactly once.
Bytes stored $= 8 \times 4 = 32$ GB.
- Why this step? Each distinct 4 GB blob is one full whole-file copy (no cross-version deltas by default).

Verify: Naive $= 48$ GB; DVC $= 32$ GB; saving $= 16$ GB $= 4 \times 4$ (exactly the four duplicate weeks eliminated). Units: GB throughout. ✓

Worked example Example 7 — Cell J (exam twist: rename vs re-content)

Exam-style. You have train.csv tracked by DVC. You run mv train.csv training.csv and then dvc add training.csv. Later a colleague swaps in a different 500 MB file but keeps the name train.csv and runs dvc add train.csv. Which action creates a new cache object — the rename, or the same-name swap? Explain using content-addressing.

Forecast: We instinctively think "new name = new file". Is DVC fooled by names at all?

The rename. Content is unchanged → MD5 unchanged. dvc add training.csv writes a new pointer file (training.csv.dvc) but its md5: equals the old one → cache already has the blob.
- Why this step? DVC keys on content, so the name is irrelevant to storage. Renaming stores 0 new bytes.
The same-name swap. New content under the old name → new MD5 → new full cache object.
- Why this step? Identical name, different content → different fingerprint → different blob. The name never enters the hash.

Answer: The rename creates no new cache object; the same-name swap creates a full new one. Names are cosmetic; hashes are truth.

Verify: Rename → new bytes $= 0$ . Swap → new bytes $= 500 MB = 524, 288, 000$ . The trap ("new name must mean new data") is defeated by content-addressing. ✓

Worked example Example 8 — Cell A + Cell G together (the full team round-trip)

Alice adds a 1 GB dataset and forgets dvc push. She commits and pushes to Git. Bob does git pull then dvc pull. What does Bob get, and where exactly does it break?

Forecast: Git pushed fine, so Bob is covered — true or false?

Alice: dvc add + git commit + git push. Git now carries the ~100-byte pointer with hash h. The 1 GB blob sits only in Alice's local cache.
- Why this step? Committing ships the ticket, not the coat (parent mistake #2).
Bob: git pull. He receives the pointer h. His cache is empty of h.
Bob: dvc pull. DVC asks the remote for h — but Alice never dvc pushed, so the remote doesn't have h either. Bob gets a "failed to download / not found" style error.
- Why this step? dvc pull reads the remote; if the blob was never uploaded, there's nothing to fetch.
Fix: Alice runs dvc push (cache → remote). Then Bob's dvc pull succeeds.

Verify: Byte path required end-to-end: Alice cache --(dvc push)--> remote --(dvc pull)--> Bob cache --(dvc checkout)--> Bob workdir. The missing dvc push breaks the very first link, so all downstream links fail. Git's success is irrelevant to the data leg. ✓

Recap

Recall One-line answer per cell

Cell A — first add stores ::: a full copy in cache; Git gets the ~100-byte pointer. Cell B — re-add unchanged file stores ::: 0 new bytes (same hash). Cell C — 1-byte edit stores ::: a full new whole-file copy (new hash, avalanche). Cell D — two identical files store ::: one blob (deduplication). Cell E — a 0-byte file gets ::: a valid MD5 and a pointer; add succeeds. Cell F — checkout with cache hit ::: copies from cache to workdir, no remote call. Cell G — checkout with empty cache ::: restores nothing; fix with dvc pull first. Cell H — dvc repro with unchanged deps ::: skips the stage (make-like). Cell I — cost = ::: (distinct hashes) × size, not (snapshots) × size. Cell J — rename vs re-content ::: rename = 0 new bytes; new content = new blob; names never enter the hash.

Connections

Parent: Data Versioning (DVC) — the concepts these examples exercise.
Content-addressable storage — why identical content collapses to one blob (Cells B, D, J).
Git version control — Git moves the pointer; every restore example splits into a Git leg and a DVC leg.
Object storage (S3/GCS) — the remote in Cells G and 8.
MLOps pipelines — the dvc repro skip logic of Cell H.
Reproducibility in ML — the payoff of binding code-hash + data-hash per commit.
Experiment tracking (MLflow) — pairs with DVC for metric + data pinning.