Chunking strategies for retrieval: Visual walkthrough

This is the visual companion to Chunking strategies for retrieval. If a word here feels unexplained (embedding, similarity, tokens), the linked prerequisite notes below build it.

Step 1 — What is a token? Draw the raw material

WHAT: Lay the whole document out as one long horizontal row of little squares. Each square is one token.

WHY: Before we can cut anything, we need a ruler. The row of tokens IS our ruler — position $0, 1, 2, \dots, N - 1$ . The total count of squares is the symbol $N$ .

PICTURE: Look at the strip below. It is $N = 12$ tokens wide. The number under each square is its position.

Figure — Chunking strategies for retrieval

Step 2 — A chunk is a window of width $c$

WHAT: Place a window (a coloured box) over the first $c$ squares of the strip: positions $0$ through $c - 1$ .

WHY: A retriever can only ever hand back one of these boxes (each box becomes one embedding vector stored in a vector database). So the box is the atom of retrieval. Its width $c$ is our choice — the central knob of chunking.

PICTURE: The blue box below covers exactly $c = 4$ tokens. Notice it starts at position $0$ and ends at position $c - 1 = 3$ ; that " $- 1$ " is because we count the first square as position $0$ , not $1$ .

Step 3 — Why we must overlap: a fact falls in the crack

WHAT: Draw two adjacent boxes touching at position $c$ (no gap, no overlap). Drop a pink answer span of $s = 3$ tokens right on top of that seam.

WHY: This is the failure we are trying to prevent. Seeing the pink span straddling the seam is the whole motivation for overlap — without this picture the formula's $o$ term looks arbitrary.

PICTURE: The pink span sits half in the left box, half in the right box. Neither box holds all of it. That is a broken span.

WHAT: Slide the second box left so it starts at position $c - o$ instead of $c$ . Now the region $[c - o, c)$ belongs to both boxes.

WHY: The shared strip is a safety net. Any answer span that fits inside a window of width $o + 1$ straddling the seam is now fully contained in at least one box. Set that safety width to cover our span length $s$ : $o + 1 \geq s ⟹ o \geq s - 1$

PICTURE: The green shaded strip is the overlap. Watch the same pink span from Step 3 — now the right box, reaching back into the green, swallows it whole.

Step 5 — The stride: how far each box advances

WHAT: Measure the gap between the start of box 1 (position $0$ ) and the start of box 2 (position $c - o$ ).

WHY: This distance is what actually eats through the strip. The width $c$ is how much each box sees; the stride $c - o$ is how much fresh ground each box conquers. Confusing the two is the classic chunking bug.

PICTURE: The yellow arrow spans from start-of-box-1 to start-of-box-2. Its length is $c - o$ .

Step 6 — Counting the boxes: assembling $M$

WHAT: Chain the boxes across the whole strip and count them. Box 1 covers $c$ tokens; each of the remaining $M - 1$ boxes adds a stride. We need the total reach to hit at least $N$ .

WHY: We want the smallest $M$ whose boxes still cover the last token — no fact left uncovered. That "smallest whole number that is big enough" is exactly what the ceiling $⌈ ⌉$ means.

PICTURE: Boxes tiled left-to-right, the first one flagged as the "startup cost" $c$ , the rest as repeated strides. The final box reaches (or passes) position $N$ .

Set the coverage requirement and solve: $c + (M - 1) (c - o) \geq N$ $(M - 1) (c - o) \geq N - c$ $M - 1 \geq \frac{N - c}{c - o}$ $M \geq \frac{N - c}{c - o} + 1 = \frac{N - c + ( c - o )}{c - o} = \frac{N - o}{c - o}$

Step 7 — The degenerate cases (never let the reader fall through)

WHAT: Push each knob to an extreme and read the formula.

WHY: A formula you trust must survive its own boundaries. Here is every corner:

The one-picture summary

One figure holds the entire derivation: a token strip of length $N$ , three overlapping windows of width $c$ , the shared green overlap $o$ , the yellow stride arrow $c - o$ , and the count $M$ boxes — with the assembled formula printed beneath.

Recall Feynman retelling — the whole walk in plain words

Picture a long paper tape, one word per square — that length is $N$ . To find facts fast you lay a small frame of width $c$ over the tape; whatever the frame shows becomes one searchable "card." But if you slide the frame in clean jumps of $c$ , a two-word fact sitting on a seam gets sliced in half and is lost forever. So you pull each next frame back a little, letting it re-read the last $o$ words of the previous one — that shared strip catches any fact of length up to $o + 1$ , so you pick $o$ at least $s - 1$ for an answer of $s$ words. Because of the sharing, each frame after the first only walks forward by $c - o$ new words — the stride. The first frame costs a full $c$ ; the rest each cost one stride. Add them up, demand you cover the last word, round up so the leftover bit still gets a card, and out drops $M = ⌈(N - o) / (c - o)⌉$ . Freeze the frame ( $o = c$ ) and it never moves — infinite cards — which is why overlap must stay smaller than the chunk.

Recall Quick self-test

What does the stride $c - o$ measure? ::: The number of new tokens each chunk adds beyond the previous one. Why is there a ceiling in $M$ ? ::: A partial final chunk still needs its own embedding vector, so we round up. Set $o = 0$ in the formula — what do you get? ::: $M = ⌈ N / c ⌉$ , the plain no-overlap tiling. Why must $o < c$ ? ::: If $o \geq c$ the stride $c - o \leq 0$ , the window never advances and $M$ blows up. Minimum overlap to keep an $s$ -token answer intact? ::: $o \geq s - 1$ .

Back to the parent: Chunking strategies for retrieval · related: Retrieval-Augmented Generation (RAG), Metadata Filtering in Retrieval.