Chunking strategies for retrieval — Hinglish notes

4.4.13 · AI-ML › Alignment, Prompting & RAG

WHAT hai chunking?

RAG pipeline kuch aisi hai: document → chunk → embed → store in vector DB → (query) embed → nearest-neighbour search → top-k chunks → LLM. Chunking bilkul shuruat mein hoti hai — baad ki saari quality isi pe depend karti hai.

WHY number-of-tokens matter karta hai (ek mini derivation)

Maano ek document mein $N$ tokens hain aur hum $c$ tokens size ke chunks mein split karte hain. Chunks ki sankhya:

$M = ⌈ \frac{N}{c} ⌉$

Ceiling kyun? Aakhri chunk partial ho sakta hai lekin phir bhi uska apna vector chahiye.

Har chunk ek vector mein embed hota hai. Toh index size $M \propto 1/ c$ ke saath badhti hai: chhote chunks ⇒ zyada vectors ⇒ zyada storage aur slow search, lekin finer resolution.

Ab probability ki ek zaruri fact ek single retrievable chunk mein salamat rahe. Agar answer $s$ tokens tak failaa hai aur hum chunk boundaries bina overlap ke rakhte hain, toh answer tab split hota hai jab koi boundary uske andar padti hai. "Boundary positions" ka woh fraction jo length- $s$ span ko todta hai:

$P (break) \approx \frac{s - 1}{c}$

Effective stride (har chunk mein naye tokens) $c - o$ hai, toh chunk count ban jaata hai:

$M = ⌈ \frac{N - o}{c - o} ⌉$

Yeh form kyun? Pehla chunk $c$ tokens consume karta hai; baad ka har chunk stride $c - o$ se aage badhta hai. $c + (M - 1) (c - o) \geq N$ solve karne par yahi formula milta hai.

Figure — Chunking strategies for retrieval

HOW: main strategies (80/20 order mein)

1. Fixed-size (token/char) chunking

Har $c - o$ tokens par overlap $o$ ke saath split karo. Simple, fast, predictable index size.

Kab: uniform prose, logs, transcripts.
Weakness: mid-sentence cut ho jaata hai → semantic blur.

2. Recursive / structure-aware chunking

Pehle sabse bade natural separator par split karne ki koshish karo (\n\n paragraphs), phir sentences, phir words, aur hard cuts tab hi karo jab koi piece $c$ se bada ho.

Kyun behtar: boundaries meaning ke saath align hoti hain, toh har chunk ek coherent idea hota hai.
Kab: articles, docs, markdown.

3. Semantic chunking

Har sentence embed karo; jab consecutive sentences ke beech cosine distance ek threshold se zyada ho jaaye (topic shift), tab naya chunk shuru karo.

Kyun: boundaries meaning follow karti hain, token counts nahi.
Cost: bahut saare extra embedding calls.

4. Document-structure chunking

Headings/sections/tables/code-blocks ko units ki tarah use karo (ek section = ek chunk). Heading path ko metadata ki tarah attach karo.

Kab: technical docs, legal contracts, wikis.

5. Small-to-big / parent-document retrieval

Chhote chunks index karo (precise matching ke liye) lekin context ke liye bada parent return karo.

Kyun: retrieval unit aur generation unit ko decouple karta hai — dono ka best milta hai.

Worked examples

Common mistakes (steel-manned)

Recall Feynman: ek 12-saal ke bacche ko samjhao

Socho ek bahut badi library book hai jisme ek amazing fact chhupa hai. Apne dost ko poori book dene ki jagah, tum use index cards mein faad dete ho — har card mein ek clear idea hota hai. Ab jab dost koi sawaal karta hai, tum jaldi se sahi card dhundh sakte ho. Agar cards bahut bade hain toh unme bahut saare ideas hote hain aur confusing lagte hain; bahut chhote hain toh har card apne aap mein samajh nahi aata. Aur tum neighbouring cards ko ek sentence share karne dete ho taaki koi fact jo do cards ke beech aata ho woh kabhi kho na jaaye. Yahi hai chunking.

Active-recall flashcards

RAG pipeline mein chunking kya hai?

Documents ko chhote pieces mein split karna jo individually embed aur index hote hain, taaki retrieval poore documents ki jagah relevant chunks return kare.

Bahut bada chunk retrieval ko kyun hurt karta hai?

Uski ek embedding kai ideas ko ek blurry vector mein average kar deti hai jo queries se weakly match karta hai, precision kam karta hai aur context tokens waste karta hai.

Bahut chhota chunk retrieval ko kyun hurt karta hai?

Surrounding context kho jaata hai, toh embedding ambiguous hoti hai aur LLM ke paas reason karne ke liye kaafi nahi hota.

N tokens par overlap o aur size c ke saath chunks ki sankhya ka formula?

M = ⌈(N - o) / (c - o)⌉

Boundary ke paas s tokens ke span ko intact rakhne ke liye minimum overlap kitna chahiye?

o \geq s - 1

Chunk starts ke beech effective stride kya hai?

c - o

(chunk size minus overlap).

FRS-DP strategies ke naam batao.

Fixed-size, Recursive/structure-aware, Semantic, Document-structure, Parent-document (small-to-big).

Semantic chunking boundaries kaise decide karta hai?

Jab consecutive sentence embeddings ke beech cosine distance ek threshold se zyada ho jaaye (topic shift), tab naya chunk shuru hota hai.

Small-to-big / parent-document retrieval kya hai?

Matching ke liye chhote precise chunks index karo lekin context ke liye unka bada parent chunk return karo.

Characters ki jagah tokens par chunk kyun karo?

Embedding models aur context windows tokens count karte hain (~4 chars/token English mein, language/code ke hisaab se alag), toh char limits model se mismatch karti hain.

Connections

Retrieval-Augmented Generation (RAG)
Text Embeddings and Cosine Similarity
Vector Databases and ANN Search
Context Window Limits of LLMs
Tokenization
Metadata Filtering in Retrieval