Reranking and hybrid search: Visual walkthrough

Step 1 — Two lists, two rulers that don't line up

WHAT. We run one query through two engines. The keyword engine (BM25) hands back a list of documents with scores. The meaning engine (bi-encoder + cosine) hands back its own list with its own scores.

WHY. We must combine them into one ordering to feed the LLM. The naive move is "add the two scores." Look at the picture and see why that is a trap.

PICTURE. On the left, BM25 scores climb high — $12.4$ , $9.1$ , $7.8$ — with no ceiling. On the right, cosine scores sit squeezed inside $[- 1, 1]$ — $0.81$ , $0.79$ , $0.77$ . If we add them, BM25's raw magnitude swamps cosine. The number $12.4$ isn't "more relevant" than $0.81$ ; they are measured on different rulers.

Step 2 — Throw away the numbers, keep the order

WHAT. We discard the raw scores entirely and remember only rank — the position of each document in each list. Rank $1$ = the top of that list, rank $2$ = second, and so on.

WHY. Rank is the one thing both engines produce on the same ruler: "1st, 2nd, 3rd…" means the same for BM25 as for cosine. It is a common currency. This is the key idea of Reciprocal Rank Fusion: fuse on rank, not on score.

PICTURE. The two coloured score-bars from Step 1 collapse into two plain numbered columns. The tall $12.4$ BM25 bar becomes just "rank 1"; the tiny-looking $0.81$ cosine bar becomes just "rank 1" too. Now they are directly comparable.

Step 3 — We need a number that shrinks as rank grows

WHAT. We want to turn a rank (1, 2, 3, …) into a contribution — a reward — such that being near the top is worth a lot and being deep down is worth almost nothing. The obvious candidate is the reciprocal $\frac{1}{rank}$ .

WHY this function and not another? We need a rule where:

rank $1$ gives the biggest reward,
rewards fall off as rank increases (rank 100 ≈ worthless),
the reward is always positive (so evidence only ever helps).

The reciprocal $\frac{1}{rank}$ does all three. But look at the picture: it falls off too violently at the very top.

PICTURE. Plot $\frac{1}{rank}$ . Rank $1 \to 1.0$ , rank $2 \to 0.5$ . That is a 50% cliff between #1 and #2 — a brutal winner-take-all. A document the two engines both rank #2 should not be crushed just because it wasn't literally #1 in either.

Step 4 — Soften the top with a constant $k$

WHAT. Add a constant $k$ to the denominator: the contribution becomes $\frac{1}{k + rank}$ . A common default is $k = 60$ .

WHY. Watch what $k$ does to the ratio between rank 1 and rank 2.

rank 1 reward \frac{1}{k + 1} vs rank 2 reward \frac{1}{k + 2}

The $k$ in the denominator is a cushion: it makes both rewards small and close together.
With $k = 0$ (plain reciprocal): ratio $= \frac{1/1}{1/2} = 2.0$ — rank 1 is worth twice rank 2.
With $k = 60$ : ratio $= \frac{1/61}{1/62} = \frac{62}{61} \approx 1.016$ — rank 1 worth only $1.6%$ more than rank 2.

So $k$ turns a cliff into a gentle slope near the top, while the tail still decays to zero.

PICTURE. Two curves overlaid: the steep $\frac{1}{rank}$ from Step 3 and the flat-topped $\frac{1}{60 + rank}$ . The new curve is almost level across ranks 1–5 (the region we care about) and still crawls toward $0$ far out.

Step 5 — Sum across retrievers: agreement wins

WHAT. A document's final fused score is the sum of its contributions over every retriever it appears in.

RRF (d) = r \in retrievers \sum \frac{1}{k + rank _{r} ( d )}

WHY sum, not max or average? Summing means a document collects reward from each list it shows up in. A doc that both engines rank decently accumulates two medium rewards; a doc only one engine found collects one. Cross-verified evidence — the same doc surfacing via both exact-match and meaning — is exactly what we should trust most.

PICTURE. Term-by-term over the formula: the $\sum$ means "walk through every retriever." Doc B, sitting at rank 2 in both lists, gets $\frac{1}{62} + \frac{1}{62}$ — two full slices. Doc A, #1 in one list but #4 in the other, gets $\frac{1}{61} + \frac{1}{64}$ . The figure shades each slice so you see two medium slices out-total one big + one small.

Step 6 — Work the numbers (the parent's example, drawn)

WHAT. Doc A: rank 1 in BM25, rank 4 in dense. Doc B: rank 2 in BM25, rank 2 in dense. Use $k = 60$ .

RRF (A) = \frac{1}{60 + 1} + \frac{1}{60 + 4} = \frac{1}{61} + \frac{1}{64} = 0.01639 + 0.01563 = 0.03202

RRF (B) = \frac{1}{60 + 2} + \frac{1}{60 + 2} = \frac{1}{62} + \frac{1}{62} = 0.01613 + 0.01613 = 0.03226

WHY B wins. A was champion in one list but mediocre in the other. B was consistently good in both. The sum rewards consistency across evidence sources over a single spectacular showing. $0.03226 > 0.03202$ — B rises to the top of the fused list.

PICTURE. Two stacked bars. A's bar = a tall orange block ( $\frac{1}{61}$ ) + a short block ( $\frac{1}{64}$ ). B's bar = two equal teal blocks ( $\frac{1}{62}$ each). B's total edges just above A's — the win is narrow, which is the point: fusion nudges, it doesn't bulldoze.

Recall Check yourself

If Doc C is rank 1 in BM25 but absent from the dense list, what is $RRF (C)$ with $k = 60$ ? $RRF (C) = \frac{1}{61} + 0 = 0.01639$ — one slice only. It loses to B ( $0.03226$ ). Absence from a list = zero contribution, no penalty, no bonus. ::: $\frac{1}{61} \approx 0.01639$ , and it loses to both A and B.

Step 7 — Edge and degenerate cases (never leave a gap)

WHAT. We check the corners so no scenario surprises you.

WHY. A formula you trust must behave sanely when inputs are extreme.

Case	What happens	Why it's fine
Doc in only one list	Gets one slice, e.g. $\frac{1}{k + rank}$ ; missing list contributes $0$	No penalty, no undeserved bonus — Step 6 recall showed this
Both retrievers identical lists	Every doc's score simply doubles uniformly	Order is unchanged; fusion is a no-op, exactly right
$k \to \infty$	Every $\frac{1}{k + rank} \to 0$ and all ratios $\to 1$	Rank differences vanish → all docs tie → fusion loses discriminating power
$k = 0$	Reverts to plain $\frac{1}{rank}$	The harsh Step 3 cliff returns — winner-take-all
Same doc, tied rank in both	e.g. rank 3 in each: $\frac{2}{k + 3}$	Fine — symmetric, well-defined

PICTURE. A little dial from $k = 0$ to $k = \infty$ : at $k = 0$ the top ranks tower over the rest (spiky); at $k = 60$ the top few are a gentle plateau (our sweet spot); at huge $k$ everything flattens into one bland line (no signal). $k = 60$ sits in the labelled "sweet spot" band.

The one-picture summary

WHAT. The whole pipeline of this page, compressed: two ranked lists → forget scores, keep rank → soften with $\frac{1}{k + rank}$ → sum per document → one fused order.

This fused top list is then handed to the cross-encoder reranker (see parent Step 4) and finally to RAG. RRF is the merge that lets one scale-free number carry the wisdom of two very different engines. For measuring whether the fused order actually improved things, see Evaluation - Recall@k and MRR.

Recall Feynman: the whole walkthrough in plain words

Two friends each rank your books. One ranks by exact words matched, the other by what the book is about. Their scores are on different scales — one shouts numbers up to 50, the other whispers numbers between 0 and 1 — so you ignore the numbers and keep only what place each book got: 1st, 2nd, 3rd. You give each placement a prize: 1st place a big prize, further-down places smaller prizes. But you don't want 1st place to crush 2nd place, so you cushion the prizes with a padding number $k$ (like 60) — now 1st and 2nd win almost the same. Finally, each book adds up all the prizes it won from both friends. A book both friends liked (two medium prizes) beats a book one friend loved but the other ignored (one big prize, one nothing). That total is the RRF score, and it's fair no matter how loudly either friend shouts.

Recall

Why can't we just add BM25 and cosine scores? ::: They live on incomparable scales — BM25 is unbounded, cosine is in $[- 1, 1]$ — so one arbitrarily dominates. RRF fuses on rank instead, which is scale-free. What does the constant $k$ do in $\frac{1}{k + rank}$ ? ::: It cushions the top: it shrinks the gap between rank 1 and rank 2 (with $k = 60$ , only ~1.6% apart vs 2× apart at $k = 0$ ), while the tail still decays to ~0. Why sum contributions across retrievers instead of taking the max? ::: Summing rewards cross-verified evidence — a doc found by both engines collects two slices and beats a doc one engine loved but the other missed. In the parent example, why does Doc B beat Doc A? ::: B is consistent (rank 2 in both → $0.03226$ ); A is a champion in one and mediocre in the other (rank 1 + rank 4 → $0.03202$ ). Fusion prefers consistency. What is $RRF$ of a doc that appears in only one list at rank 1 ( $k = 60$ )? ::: $\frac{1}{61} \approx 0.01639$ — a single slice, no penalty for the absent list.