Foundations — Retrieval-Augmented Generation (RAG) architecture

This page assumes nothing. If the parent note RAG architecture used a symbol, we build it here from the ground up, in an order where each idea leans only on the ones before it.

0. What is a vector? (the atom of everything here)

Before any RAG formula, you must be comfortable with one picture: a vector is an arrow with a tail at the origin and a tip somewhere in space.

• The picture: $(3,2)$ says walk 3 right, then 2 up, and draw an arrow from where you started to where you ended.
• Why RAG needs it: RAG converts every chunk of text and every question into one of these arrows (this conversion is called embedding). Once text is an arrow, "is this document about the same thing?" becomes a geometry question we can actually compute.

1. Length of a vector: the norm $\Vert \mathbf{q}\Vert$

The parent note writes $\Vert \mathbf{q}\Vert$ and calls it "magnitude". Let us earn that symbol.

• Why the square root of squares? Look at the figure: the arrow is the hypotenuse of a right triangle whose legs are $q_1$ and $q_2$. Pythagoras says (leg)${}^2$ + (leg)${}^2$ = (hypotenuse)${}^2$, so the hypotenuse is $\sqrt{q_1^2+q_2^2}$. The norm is the vector's length via the same triangle you met in school.
• Example: $\Vert (3,4)\Vert =\sqrt{9+16}=\sqrt{25}=5$.
• Why RAG cares: a longer document tends to produce a longer arrow. The parent note's whole argument for cosine over dot product is "cancel out this length so it can't cheat". You cannot understand that argument without knowing what length means.

2. The dot product $\mathbf{q}\cdot \mathbf{d}$

This is the single most important operation in the whole topic. The parent note uses $\mathbf{q}\cdot \mathbf{d}$ everywhere.

• Why two formulas matter: the left one is what a computer calculates (fast: multiply and add). The right one is what it means (length × length × alignment). RAG needs both: the fast side to run, the meaningful side to justify ranking by it.
• The picture: the dot product is big when the arrows point the same way, zero when they are perpendicular (90° apart, no shared direction), and negative when they point opposite ways. That "same-way-ness" is precisely "same topic".
• Why not just use this raw number? Because it multiplies in $\Vert \mathbf{q}\Vert \Vert \mathbf{d}\Vert$ — the lengths. A long document (big $\Vert \mathbf{d}\Vert$) scores high just for being long. That is the flaw cosine will fix next.

3. The angle $\theta$ and cosine $\cos \theta$

The parent's headline formula is $\cos \theta =\frac{\mathbf{q}\cdot \mathbf{d}}{\Vert \mathbf{q}\Vert \Vert \mathbf{d}\Vert }$. Now it is fully earned.

4. Sum $\sum$ and the retriever/generator split

The parent note's definition uses $\sum _{z\in \mathcal{Z}}$. Two symbols to unpack.

• Picture: a stack of index cards $\mathcal{Z}$; you flip through each card $z$, score it, and total the scores.
• Why RAG needs it: RAG never knows which single passage holds the answer, so it weighs all of them and sums their contributions — that is what $p(y\mid x)={\sum }_{z}p(z\mid x)p(y\mid x,z)$ says. The two pieces:
  • $p(z\mid x)$ — the retriever: "how likely is passage $z$ the right one for query $x$?"
  • $p(y\mid x,z)$ — the generator: "given the query and that passage, how likely is answer $y$?"
• $p(\cdot \mid \cdot )$ reads "probability of the left thing given the right thing is known". The vertical bar $\mid$ means "given / assuming".

5. Turning scores into probabilities: softmax and $e^x$

The parent's Example 2 uses $p(z_i\mid x)=\frac{e^{s_i}}{{\sum }_j{e}^{s_j}}$. Let us define $e$ and why it appears.

Let us verify the parent's Example-2 arithmetic in the checklist below.

6. How the foundations feed the topic

Read top to bottom: the vector is the atom; norm and dot product are built from it; cosine is dot product with norms divided out; softmax (built on $e^x$) and the sum turn scores into the probability the parent note models. All roads meet at the RAG pipeline.

Equipment checklist

Cover the right side and answer aloud. If any stumps you, re-read that section before tackling the parent note.