Chunking strategies for retrieval: Foundations

Before you can size a chunk, you must know what the ruler measures, what a "piece" is, and what "shared edges" mean. This page builds every one of those symbols from the ground up.

0. The raw object: a document as a line of tokens

Everything starts with the text. But computers and embedding models do not see letters — they see tokens.

Picture the whole document stretched out as a single horizontal ribbon, chopped into little cells — each cell is one token. This ribbon is the stage on which every other symbol lives.

Figure — Chunking strategies for retrieval

1. $N$ — the length of the document

Picture: the length of the whole ribbon, measured in cells. That's it. It is the thing we are about to cut up.

2. A chunk — one piece of the ribbon

Picture: take scissors, cut the ribbon into segments. Each segment is a chunk. The retriever's job is to find the one segment whose meaning matches your question — so the segment is the atom of retrieval.

Why does the topic need this word? Because the parent's whole thesis — "a retriever can only return the units you gave it" — is a statement about chunks. The chunk is the unit.

3. $c$ — the chunk size

Picture: the width of your scissors' stride — how far along the ribbon you walk before making a cut.

4. $M$ — how many chunks you get

The strange hat symbol $⌈ ⌉$ is the ceiling: round up to the next whole number.

Picture: the number of segments lying on the table after you finish cutting.

Why the topic needs $M$ : it is your index size. More chunks = more vectors = more storage and slower nearest-neighbour search in the vector database. Since $M \propto 1/ c$ , halving chunk size roughly doubles your index.

5. The problem $M$ ignores: facts get cut

If you cut with plain scissors, a boundary can land in the middle of a sentence. Look at the red cut below slicing "melts down at 1132°C" into two chunks — neither chunk now contains the whole fact, so neither can be retrieved to answer it.

Picture: a coloured band of $s$ cells somewhere on the ribbon. If a cut line falls inside the band, the fact is broken across two chunks.

6. $o$ — overlap, the shared edge

Picture: instead of butt-joining the segments, you let each pair of neighbours overlap like roof shingles by $o$ cells. Any small band sitting on the seam is now fully contained in at least one shingle.

7. Stride $c - o$ — how far each cut advances

Picture: the shingles overlap by $o$ , so walking from one shingle's start to the next you only travel $c - o$ cells.

8. Metadata — the label on each card

Picture: a sticky note on each shingle saying "from §4.2, Reactor Safety". The retriever can pre-filter to only shingles with the right label before comparing meanings — a second, cheaper handle beyond similarity.

The legal ranges at a glance

Every symbol has a domain; violate it and the machinery breaks in a specific, predictable way.

Symbol	Legal range	What breaks if violated
$N$	$N \geq 1$ (integer)	nothing to chunk
$c$	$c \geq 1$ (integer)	empty chunks, no vector
$o$	$0 \leq o < c$	stride $\leq 0$ : chunks never advance
$s$	$s \leq c$ to be containable	answer can't fit any single chunk

Together these feed Chunking strategies for retrieval: tokens set the ruler; $N$ , $c$ , $s$ describe the cut; $o$ and the stride $c - o$ protect facts and fix $M$ ; metadata adds a second filter. Every strategy in the parent note is a different rule for where to place those cuts — but they all obey these same ranges.

A tiny worked check of the machinery

Equipment checklist

You are ready for the main note only if you can answer each without peeking.

What does a single token roughly correspond to in English?

About 4 characters, or ¾ of a word — the unit both the embedding model and context window count in.

What is a chunk, physically?

A contiguous stretch of the token-ribbon treated as one indexed unit with exactly one embedding vector.

What does

N

measure and what is its range?

The total number of tokens in the document;

N \geq 1

What does

c

control, its range, and the tradeoff it dials?

Chunk size in tokens, integer

c \geq 1

; it trades retrieval precision (small

c

) against context completeness (large

c

Why does

M

use a ceiling?

A leftover partial chunk still needs its own vector, so you round up — you never discard text.

Why is

M

roughly

1/ c

, and why does that matter?

Smaller chunks make more vectors, growing index size and slowing nearest-neighbour search.

What is

s

and what must hold for a chunk to contain it?

The token length of a fact you want retrievable intact; you need

s \leq c

or no single chunk is wide enough.

What assumption underlies

P (break) \approx (s - 1) / c

That the span's start position is uniformly distributed relative to the cut grid (uniform modulo

c

State the overlap guarantee, its range, and its reason.

o \geq s - 1

with

0 \leq o < c

; the

o

shared tokens let any window up to

o + 1

long straddle a seam inside one chunk.

What is the stride, how does it differ from

c

, and why must

o < c

c - o

, the new tokens per chunk; it equals

c

only at zero overlap, and

o < c

keeps the stride positive so chunks advance.

What does metadata add that embeddings don't?

A structured label (heading, section, date) you can filter on cheaply before comparing meanings.