Noesis
4.4.12 · D5Alignment, Prompting & RAG

Question bank — Vector databases and embeddings

1,497 words7 min read↑ Back to topic

This is the conceptual companion to the parent topic. Every item links back to one idea there: meaning becomes geometry, and search becomes "find the nearest points."

True or false — justify

A vector with more components (higher ) always stores more meaning.
False. Dimension is a design choice of the model, not a meaning-meter; a well-trained -dim embedder can beat a poorly-trained -dim one. Extra unused dimensions add cost and worsen the Curse of Dimensionality, not understanding.
Cosine similarity of means the two texts are word-for-word identical.
False. Cosine means the vectors point the same direction, i.e. the model judged the meanings identical (e.g. "car" and "automobile"). Identical strings give cosine , but so can perfect paraphrases.
If two vectors are unit-normalized, their dot product is already the cosine similarity.
True. With the formula has denominator , so — the reason databases pre-normalize once and then use the cheap Dot Product.
Euclidean distance and cosine similarity always rank neighbours the same way.
False in general, true only for unit-normalized vectors, where . On raw (unnormalized) vectors, Euclidean is dominated by length, so a long document can look "far" despite matching topic.
A cosine similarity of means the two texts are opposites.
False. Cosine means the vectors are orthogonal (90° apart) → unrelated, not opposite. Opposite meaning would push toward ; in practice trained embedders rarely reach .
Doubling every number in a vector doubles its cosine similarity with another vector.
False. Scaling changes the length but not the direction, and cosine only sees direction. vs still gives cosine — this is exactly why we divide by the norms.
ANN search will always return the exact true nearest neighbours, just faster.
False. The "A" is Approximate — it trades a small chance of missing a true neighbour for near-logarithmic speed. Exact k-NN guarantees the top- at cost; ANN usually finds them but does not promise it.
You can safely embed your documents with model X and your queries with model Y as long as both output -dim vectors.
False. Matching dimension is not matching coordinate space. Two models place "dog" at different points, so cross-model cosine is meaningless noise — always embed query and documents with the same model.

Spot the error

"Keyword search already finds synonyms, so embeddings are overkill."
The error: keyword search matches shared characters, and "car"/"automobile" share almost none. Embeddings exist precisely because semantic closeness ≠ string overlap.
"To compare meanings I'll just check if the two vectors have the same length."
Length () encodes magnitude, not meaning-direction. Two unrelated texts can share a norm; you must compare angle, which is what cosine does.
"Cosine similarity = 1 − Euclidean distance."
Wrong formula. The correct relation for unit vectors is ; and cosine distance is , which is not the same object as Euclidean distance.
"I'll make chunks as large as possible so no context is ever lost."
One vector must summarize the whole chunk; a huge chunk mixes many topics into one muddled direction, hurting retrieval. Good chunking is ~100–300 tokens with slight overlap.
"HNSW searches by scanning every node in the bottom layer."
That would be a full scan, defeating the point. HNSW starts at a sparse top layer, greedily hops toward closer nodes, then descends layer by layer, touching only a tiny fraction of nodes → near-logarithmic cost.
"Since the dot product already contains , cosine similarity adds nothing."
The raw Dot Product is — it mixes magnitude with direction. Cosine isolates by dividing out the norms; that isolation is the whole point for meaning-matching.
"Normalizing vectors throws away important information, so never do it."
For semantic matching we want to throw away magnitude and keep direction only. Normalization is deliberate, and it lets the database replace division-heavy cosine with a cheap dot product.

Why questions

Why divide the dot product by the two norms at all?
To strip out length so only the angle (meaning-direction) survives, giving a bounded score in where a long and a short document about the same topic still match.
Why is a plain SQL WHERE clause the wrong tool for a vector database?
WHERE matches exact values; retrieval needs approximate geometric proximity in high-dimensional space, which requires specialized indexes like HNSW.
Why do embedding models use Contrastive Learning instead of hand-designed numbers?
We can't hand-write numbers per word. Contrastive training pulls related texts together and pushes unrelated apart, letting the model discover a geometry where similarity = closeness.
Why does exact k-NN become impractical at billion-scale?
Exact search compares the query to every stored vector: . For that's a billion dot products per query — far too slow, motivating ANN.
Why must the same model embed both query and documents?
Each model defines its own coordinate space; cosine similarity is only meaningful when both vectors live in the same space, otherwise the angle is comparing incomparable axes.
Why do we store the original text as metadata alongside each vector?
The vector is only a numeric address; retrieval returns vectors, but the LLM needs the actual passage to read. The stored text is what gets fed into the prompt for Retrieval-Augmented Generation.
Why does retrieval pick the highest cosine, not the lowest?
Higher cosine = smaller angle = closer meaning-direction. The lowest cosine would be the most unrelated chunk, the opposite of what we want.

Edge cases

What is the cosine similarity of a vector with itself?
Exactly : the angle is , so . This is the maximum possible score and the sanity-check every embedder must pass.
What happens to cosine similarity if one vector is the zero vector ?
It is undefined — the norm is , so we divide by zero, and a zero vector has no direction. Systems must guard against empty/degenerate embeddings.
Can cosine similarity ever be exactly for real embeddings?
Mathematically yes (perfectly opposite directions), but trained text embedders rarely produce it, since "opposite meaning" isn't a clean geometric reflection — most unrelated pairs cluster near , not .
What if two chunks are near-duplicates — will retrieval waste slots on both?
Yes; both score high cosine and can fill your top- with redundant content. This is why pipelines add de-duplication or diversity re-ranking after the k-NN search.
As dimension grows very large, what happens to distances between random points?
They concentrate — most pairwise distances become nearly equal, so "nearest" is barely nearer than "farthest." This is the Curse of Dimensionality, and it's why good embeddings and ANN indexes matter.
What is the cosine similarity between two orthogonal vectors, e.g. and ?
Exactly : their dot product is and the angle is , the canonical "unrelated" case.
What happens if a query is embedded but the database index was built with unnormalized vectors while the query is normalized?
The dot-product shortcut breaks — you'd be comparing (query) against (docs), so rankings get skewed by document length. Normalize both sides consistently.
Recall One-line self-test

If you can explain why cosine ignores length but Euclidean doesn't, and why ANN is "approximate," you've absorbed the two deepest traps on this page. Cosine ignores length because ::: it divides by both norms, leaving only (pure direction). ANN is approximate because ::: it trades a tiny chance of missing a true neighbour for near-logarithmic speed instead of .

Connections