Vector databases and embeddings — Hinglish notes

4.4.12 · AI-ML › Alignment, Prompting & RAG

Embeddings ki zaroorat kyun hai?

WHAT ek embedding kya hai: ek function $f : text \to R^{d}$ jo kisi bhi piece of text ko ek fixed-length vector mein map karta hai (jaise $d = 384, 768, 1536$ ). Ise is tarah train kiya jaata hai ki similar meaning wale texts ke vectors similar directions mein point karein.

HOW yeh seekha jaata hai: models (jaise sentence-transformers, OpenAI text-embedding-3) huge corpora par train kiye jaate hain taaki related sentences paas aayein aur unrelated ones door jayein (contrastive learning). Hum numbers hand-design nahi karte — model khud inhe discover karta hai.

Closeness measure karna: first principles se

Hum chahte hain ek number jo bataaye "vectors $a$ aur $b$ kitne similar hain?"

Cosine similarity ka derivation. Hum chahte hain ek aisi measure jo length ko ignore kare aur sirf direction pe dhyan de (dogs ke baare mein ek lamba document aur ek chhota document phir bhi match hone chahiye). Upar wali dot product identity se shuru karo aur $cos θ$ ke liye solve karo:

$sirf direction cos θ = \frac{a \cdot b}{∥ a ∥ ∥ b ∥}$

Yahi cosine similarity hai. Norms se divide kyun karte hain? Magnitude hatane ke liye taaki sirf angle (meaning-direction) bacha rahe. Range: $[- 1, 1]$ ; $1$ = identical direction, $0$ = orthogonal (unrelated), $- 1$ = opposite.

Figure — Vector databases and embeddings

Vector database KIYA hai?

Normal database kyun nahi? SQL ka WHERE exact values match karta hai. Yahan hume high-dimensional vectors par approximate geometric proximity chahiye, jiske liye specialized indexes zaroori hain.

Exact search ka curse

Exact k-NN saare $N$ vectors scan karta hai: cost $O (N \cdot d)$ . $N = 1 0^{9}$ ke liye yeh har query par bahut slow hai.

ANN kaise kaam karta hai — HNSW (popular wala): ek Hierarchical Navigable Small World graph banao. Nodes = vectors; edges nearby ones ko multiple layers mein connect karte hain (skip-list ki tarah). Query sparse top layer se shuru hoti hai, greedily closer neighbours ki taraf hop karta hai, phir refine karne ke liye layer by layer neeche aata hai. Yeh logarithmic-ish search deta hai.

Full pipeline (yahi 80/20 core hai)

Documents ko passages mein Chunk karo (taaki har vector ek focused idea represent kare).
Har chunk ko Embed karo → vector + original text (metadata) store karo.
User query ko usi same model se Embed karo.
DB mein top- $k$ nearest chunks ke liye Search karo.
Un chunks ko ek LLM ko do (yahi step = RAG, dekho Retrieval-Augmented Generation).

Worked examples

Recall Feynman: 12-saal ke bachche ko explain karo

Socho har sentence ek giant star-map par ek dot hai. Same meaning wale sentences neighbouring stars ki tarah paas-paas rakhе jaate hain. Tera question answer karne ke liye, computer teri query ko bhi ek dot bana deta hai, phir bas uske aas-paas ke nearest stars dhundh ke padh leta hai. "Vector database" woh telescope hai jo bina poore sky ke har star check kiye nearby stars super fast dhundh leta hai.

Common mistakes

Flashcards

Embedding kya hai?

Ek learned function jo text ko

R^{d}

mein fixed-length vector mein map karta hai jahan semantic similarity geometric closeness se correspond karti hai.

Raw dot product ki jagah cosine similarity kyun use karein?

Yeh vector magnitude ko divide out karta hai taaki sirf direction (meaning) matter kare; document ki length topical match ko affect nahi karni chahiye.

Cosine similarity ka formula

\frac{a \cdot b}{∥ a ∥∥ b ∥} = cos θ

, range

[- 1, 1]

Vector database kya karta hai?

Embeddings ko index karta hai aur diye gaye query vector ke liye fast (approximate) k-nearest-neighbour queries ka jawab deta hai.

ANN kya hai aur kyun use karein?

Approximate Nearest Neighbour —

O (N d)

exact scan ke comparison mein thodi accuracy trade karke near-logarithmic speed milti hai.

HNSW search kaise kaam karta hai?

Multi-layer navigable small-world graph; sparse top layer se dense bottom layer tak greedy hop karte hue closer neighbours ki taraf jaata hai.

Query aur documents ke liye same embedding model kyun use karna chahiye?

Alag models alag coordinate spaces mein rehte hain, isliye cross-model cosine similarity meaningless hoti hai.

Agar vectors unit-normalized hain toh dot product equal hota hai

cos θ

(cosine similarity directly).

Unit vectors ke liye Euclidean aur cosine ka relation

∥ a - b ∥^{2} = 2 - 2 cos θ

, toh dono identically rank karte hain.

5-step retrieval pipeline

Chunk → Embed → Embed query → Search (k-NN) → Rank/LLM ko do (RAG).

Connections

Retrieval-Augmented Generation — vector search "retrieval" engine hai.
Prompt Engineering — retrieved chunks prompt context mein inject kiye jaate hain.
Cosine Similarity · Dot Product · Vector Norms
Contrastive Learning — embedding models kaise train hote hain.
HNSW and ANN Indexes · Chunking Strategies
Curse of Dimensionality — high $d$ mein exact search kyun mushkil ho jaata hai.