Worked examples — Reranking and hybrid search

This page is the "run every case through the machine" companion to Reranking and hybrid search. We will not introduce new theory — we will exercise every corner of it: every fusion tie, every degenerate rank, every zero, and the limits where the formulas do something surprising.

Before any number appears, three plain-word definitions so no symbol is used unearned:

Before any number appears, one reminder of the two knobs we'll turn over and over:

Recall The two formulas we are exercising

RRF (rank-fusion): $\text{RRF}(d)=\sum _r\frac{1}{k+{\text{rank}}_r(d)}$ — uses only positions, never raw scores. BM25 saturation: $\text{score}=\text{IDF}\cdot \frac{f(k_1+1)}{f+k_1\left(1-b+b\frac{|D|}{\text{avgdl}}\right)}$ — term frequency $f$ with diminishing returns; $\text{IDF}$, $|D|$ (doc length) and $\text{avgdl}$ (average doc length) defined above. Cosine: $\cos \theta =\frac{\mathbf{q}\cdot \mathbf{d}}{\Vert \mathbf{q}\Vert \Vert \mathbf{d}\Vert }\in [-1,1]$ — the semantic score.

---

The scenario matrix

Every worked example below is tagged with the cell it covers. Read this table first — it is the whole space of things this topic can throw at you.

#	Cell (case class)	What is being stressed	Example
C1	Rank tie / cross-verification	doc in both lists beats a #1 in one	Ex 1
C2	Only-one-list doc	a doc missing from a retriever entirely	Ex 2
C3	Choice of $k$ (limit $k\to 0$ and $k\to \infty$)	how smoothing changes winners	Ex 3
C4	BM25 TF saturation limit ($f\to \infty$)	diminishing returns ceiling	Ex 4
C5	BM25 length normalisation, $b=0$ vs $b=1$	long-doc penalty extremes	Ex 4
C6	IDF sign: rare vs super-common term	when IDF is large, small, or ≈0	Ex 5
C7	Cosine signs: aligned / orthogonal / opposite	$+1$, $0$, $-1$ cases; the zero vector	Ex 6
C8	Real-world word problem	full hybrid→rerank pipeline decision	Ex 7
C9	Exam twist: "bigger top-k vs rerank"	lost-in-the-middle trade-off, numeric	Ex 8
C10	Degenerate: rerank cannot recall	doc absent from top-100	Ex 9

Prereqs if any cell feels shaky: BM25 and TF-IDF, Embeddings and Bi-encoders, Cross-encoders and Sentence Transformers, Evaluation - Recall@k and MRR.

---

Example 1 — C1 · The cross-verified doc wins a tie

1. Write each doc's contributions. Why this step? RRF is a sum of $\frac{1}{k+\text{rank}}$ over the lists the doc appears in — nothing else matters, so we just list the ranks. $\text{RRF}(A)=\frac{1}{60+1}+\frac{1}{60+5}=\frac{1}{61}+\frac{1}{65}$ $\text{RRF}(B)=\frac{1}{60+3}+\frac{1}{60+3}=\frac{2}{63}$

2. Evaluate. Why this step? We need the actual numbers to compare — the fractions are close, so intuition can mislead. $\text{RRF}(A)=0.016393+0.015385=0.031778$ $\text{RRF}(B)=0.015873+0.015873=0.031746$

3. Compare. $0.031778>0.031746$, so A wins here — barely. Why this step? This is the deliberate contrast with the parent note's tiny example (where B won). The difference: there A was rank 4 in dense; here A is rank 5 but its #1 is strong enough. RRF is a genuine trade-off, not a rule of thumb.

---

Example 2 — C2 · A doc that only one retriever found

1. Model "absent" correctly. Why this step? If a doc is not in a retriever's returned list, it contributes nothing from that retriever (the term is simply omitted, not set to some huge rank). This is the crux of the case. $\text{RRF}(C)=\frac{1}{60+1}+0=\frac{1}{61}=0.016393$

2. Sum D's two contributions. $\text{RRF}(D)=\frac{1}{62}+\frac{1}{62}=\frac{2}{62}=0.032258$ Why this step? D is cross-verified — present in both lists — so it accumulates two terms.

3. Compare. $0.032258>0.016393$, so D wins comfortably. Why this step? This is the whole philosophy of hybrid: agreement across modalities (lexical and semantic) beats a lone strong signal. A doc found by both retrievers is safer evidence.

---

Example 3 — C3 · The limit behaviour of $k$

1. Case $k=0$ (winner-take-all). Why this step? Setting $k=0$ makes rank-1 worth $\frac{1}{1}=1$, dwarfing everything. This exposes what $k$ was protecting us from. $\text{RRF}(A)=\frac{1}{1}+\frac{1}{5}=1.2000,\text{RRF}(B)=\frac{1}{3}+\frac{1}{3}=0.6667$ A wins by a landslide — a single #1 dominates.

2. Case $k$ large, e.g. $k=1000$. Why this step? Large $k$ flattens the gap between ranks, so what matters is how many lists you appear in and their average rank, not the exact position. $\text{RRF}(A)=\frac{1}{1001}+\frac{1}{1005}=0.00199403$ $\text{RRF}(B)=\frac{1}{1003}+\frac{1}{1003}=0.00199402$ The two are a dead heat to five decimals — A still edges B by less than $10^{-8}$, so the near-tie is the real lesson: rank position has almost stopped mattering.

3. Read the limit. As $k\to \infty$, every $\frac{1}{k+\text{rank}}\to \frac{1}{k}$, so a doc in $L$ lists scores $\approx \frac{L}{k}$ — only the count of appearances survives, ranks vanish. Why this step? This tells you why the default $k=60$ is a compromise: small enough that rank order still matters, large enough that rank-1 isn't dictatorial. Both A and B appear in $L=2$ lists, so at the infinite-$k$ limit they become exactly equal.

---

Example 4 — C4 & C5 · BM25 saturation and the two length extremes

1. Set up the saturating fraction with $b=0$. Why this step? With $b=0$ the length term collapses to $1-b+b(\dots )=1$, isolating pure TF saturation. $s(f)=\text{IDF}\cdot \frac{f(k_1+1)}{f+k_1}=2\cdot \frac{2.5f}{f+1.5}$

2. Evaluate the ladder. $s(1)=2\cdot \frac{2.5}{2.5}=2.000,s(5)=2\cdot \frac{12.5}{6.5}=3.846$ $s(100)=2\cdot \frac{250}{101.5}=4.926,s(\infty )=2\cdot 2.5=5.000$ Why this step? Watch the diminishing returns: going $1\to 5$ adds $1.85$; $5\to 100$ adds only $1.08$; the hard ceiling is $\text{IDF}\cdot (k_1+1)=5$. That is the "50th cat tells us nothing" effect made numeric.

3. Now length, $b=1$, $f=5$, doc twice as long. Why this step? $b=1$ turns on full length normalisation; a long doc (its $|D|$ is twice $\text{avgdl}$) is "diluting" its 5 hits. $s=2\cdot \frac{5\cdot 2.5}{5+1.5(1-1+1\cdot 2)}=2\cdot \frac{12.5}{5+3}=2\cdot \frac{12.5}{8}=3.125$ Compare to the $b=0$ value at $f=5$ which was $3.846$: the long doc is penalised from $3.846$ down to $3.125$.

The figure below plots this. The lavender curve is $b=0$ (length ignored) and the coral curve is $b=1$ on a doc twice the average length — notice how coral sits below lavender everywhere (the length penalty) and how both flatten toward the mint dashed ceiling at $5$ as $f$ grows. The two slate dots mark $f=5$ on each curve, the exact numbers from step 3.

---

Example 5 — C6 · IDF across the sign spectrum: rare, common, and ubiquitous terms

1. Rare term, $n=10$. Why this step? This is the discriminating case — the whole point of IDF is to reward rarity. $\ln \frac{1,000,000-10+0.5}{10+0.5}+1=\ln \frac{999,990.5}{10.5}+1=\ln (95,237.2)+1\approx 11.464+1=12.464$

2. Common term, $n=500,000$ (half the corpus). $\ln \frac{500,000.5}{500,000.5}+1=\ln (1.0000)+1=0+1=1.000$ Why this step? At exactly half the corpus the log argument is $1$, so $\ln =0$ and IDF flattens to the $+1$ floor. This term barely discriminates.

3. Ubiquitous term, $n=999,990$. $\ln \frac{10.5}{999,990.5}+1=\ln (0.0000105)+1=-11.464+1=-10.464$ Why this step? The log goes negative — the raw BM25 IDF can be negative for terms in almost every doc. This is why production BM25 often floors IDF at $0$: a "stopword-like" term shouldn't actively subtract from relevance.

---

Example 6 — C7 · Cosine at every sign, and the degenerate zero vector

1. Compute $\Vert \mathbf{q}\Vert$ once. Why this step? Cosine normalises out magnitude — only direction matters — so we need the length. $\Vert \mathbf{q}\Vert =\sqrt{3^2+4^2}=\sqrt{25}=5$.

2. ${\mathbf{d}}_1=(6,8)$ — same direction. $\cos =\frac{3\cdot 6+4\cdot 8}{5\cdot \sqrt{36+64}}=\frac{18+32}{5\cdot 10}=\frac{50}{50}=1$ Perfect semantic alignment → score $+1$. Why: ${\mathbf{d}}_1=2\mathbf{q}$, so same direction regardless of scale.

3. ${\mathbf{d}}_2=(-4,3)$ — perpendicular. $\cos =\frac{3(-4)+4(3)}{5\cdot \sqrt{16+9}}=\frac{-12+12}{5\cdot 5}=\frac{0}{25}=0$ Unrelated meaning → score $0$. Why: dot product zero means the vectors share no directional overlap.

4. ${\mathbf{d}}_3=(-3,-4)$ — opposite. $\cos =\frac{3(-3)+4(-4)}{5\cdot 5}=\frac{-9-16}{25}=\frac{-25}{25}=-1$ Anti-aligned → score $-1$ (the floor of the range). Why: ${\mathbf{d}}_3=-\mathbf{q}$.

5. ${\mathbf{d}}_4=(0,0)$ — the degenerate case. Why this step? An empty/padding chunk can embed near-zero. The denominator has $\Vert {\mathbf{d}}_4\Vert =0$, so cosine is $\frac{0}{0}$ — undefined. In code this is a divide-by-zero. Fix: guard with an $ϵ$ or drop zero-norm vectors before ranking.

The figure shows all four as arrows from the origin against the lavender query arrow $\mathbf{q}=(3,4)$: the mint doc points the same way (score $+1$), the butter doc is at a right angle (score $0$), the coral doc points in exactly the opposite direction (score $-1$), and the slate dot at the origin is the zero vector — it has no direction at all, so its cosine is undefined. Reading the picture, the closer a doc arrow's direction is to the lavender query arrow, the higher its cosine — that single geometric fact is the whole of dense scoring.

---

Example 7 — C8 · Real-world pipeline decision

1. RRF each chunk. $\text{RRF}(P)=\frac{1}{61}+\frac{1}{68}=0.016393+0.014706=0.031099$ $\text{RRF}(Q)=\frac{1}{66}+\frac{1}{61}=0.015152+0.016393=0.031545$ $\text{RRF}(R)=\frac{1}{62}+\frac{1}{90}=0.016129+0.011111=0.027240$ Why this step? Fusion ranks $Q>P>R$. Q wins because it's strong in both lists, even without the literal code.

2. Cross-encoder pass (precision step). Why this step? Cross-encoders read query+chunk jointly. R contains AUTH_88 but is about a build tag, not about resetting MFA — joint attention detects the mismatch and scores R low despite the exact token. P and Q both genuinely answer; Q gives the actual reset steps. Expected reranked order: Q > P > R.

3. Decision. Send Q as top-1, P as top-2, drop R. Why this step? This is the whole thesis of the parent note in one example — lexical recall keeps P and R in the pool, semantic recall surfaces Q, fusion prefers the cross-verified Q, and the reranker punishes R's surface-only match.

---

Example 8 — C9 · Exam twist: "just raise top-k into the LLM"

1. Usefulness with reranking, $p=4$. Why this step? Reranking places the true answer near the top, so it sits at a low position. $u_4=\frac{1}{1+0.15\cdot 4}=\frac{1}{1.6}=0.6250$

2. Usefulness without reranking, $p=40$. Why this step? A raw large top-k dumps the answer wherever retrieval happened to rank it — often deep in the middle. $u_{40}=\frac{1}{1+0.15\cdot 40}=\frac{1}{7}=0.142857$

3. Fraction lost. $1-\frac{u_{40}}{u_4}=1-\frac{0.142857}{0.6250}=1-0.228571=0.771429\approx 77.1\%$ Why this step? This is the numeric spine of the lost-in-the-middle mistake — a bigger, unranked context can throw away ~77% of the answer's usefulness, plus adds latency and cost.

---

Example 9 — C10 · The degenerate limit: rerank can never recall

1. Recall@100. Why this step? Recall@k = (relevant items in top-k) / (total relevant). The gold chunk is at 137, outside 100. $\text{Recall@100}=\frac{0}{1}=0$

2. Best possible Recall@5 after reranking. Why this step? A reranker can only reorder its input set. The gold chunk was never in the input, so it cannot appear in any reranked output. ${\text{Recall@5}}_{\text{rerank}}=\frac{0}{1}=0$ No matter how good the cross-encoder is, $0$ is the ceiling.

3. The lesson. Reranking is a precision operation on a fixed pool; it has zero power to recall a missing document. Why this step? This is the fourth parent-note mistake as a hard number: invest in retrieval recall (hybrid, better chunking, larger candidate pool) before spending on reranking.

---

Recall check

Recall When does a lone rank-1 doc lose to a rank-3 doc in both lists?

When the two mid-rank contributions sum above the single strong one — i.e. cross-verification beats a spike once $k$ is large enough to soften rank-1's dominance (Ex 1–3).

Recall What is the hard ceiling of a BM25 term contribution as

$f\to \infty$? $\text{IDF}\cdot (k_1+1)$ — term frequency saturates, so the 100th occurrence adds almost nothing (Ex 4).

Recall What do

$|D|$ and $\text{avgdl}$ mean in BM25? $|D|$ = length (token count) of the document being scored; $\text{avgdl}$ = average token count across the corpus. Their ratio says how long this doc is versus typical (Ex 4).

Recall Why is cosine undefined for a zero-norm document embedding?

The denominator $\Vert \mathbf{q}\Vert \Vert \mathbf{d}\Vert$ becomes $0$, giving $\frac{0}{0}$; guard with $ϵ$ or drop zero vectors (Ex 6).

Recall Can a perfect reranker fix a document that retrieval placed at rank 137 of a top-100?

No — reranking only reorders its input pool; recall must be fixed at retrieval time (Ex 9).

Back to parent: Reranking and hybrid search.