The one-query bottleneck
Plain RAG embeds the user’s question into a single vector, finds the nearest passages, and hands them to the model. Everything downstream depends on that one embedding landing in the right neighborhood. When it does not — because the question is terse, uses different vocabulary than the documents, or conflates two sub-needs — the retriever quietly returns plausible-but-wrong passages and the generator answers confidently from them.
The failure is a recall problem, not a ranking one. If the right passage never enters the candidate set, no reranker can rescue it. RAG-Fusion attacks recall at the source: instead of trusting one query to cover the intent, it spreads several queries across the semantic space around the question, so a passage that any reasonable phrasing would have surfaced gets a chance to appear. You are trading a little extra retrieval work for a much lower probability that the answer was simply never in the room.
Step one: generate multiple sub-queries
The first move is query expansion by the language model itself. Given the original question q, you prompt the model to produce n alternative phrasings — typically 3 to 5 — that restate the same information need from different angles. A question like ‘how do isolates keep cold starts low?’ might spawn ‘what is a V8 isolate,’ ‘why are container cold starts slow,’ and ‘startup latency of edge runtimes.’
The goal is controlled diversity. You want phrasings that use different vocabulary and emphasize different facets, because each one will retrieve a partially different set of documents. You do not want the model to drift off-topic or invent sub-questions the user never asked. A tight prompt — ‘rewrite this query in N distinct ways that preserve the original intent’ — plus a low-to-moderate temperature is the usual recipe. The original query is normally kept in the set too, so the user’s literal phrasing still competes.
Step two: one ranked list per sub-query
Each of the n sub-queries is run through the retriever independently, producing its own ranked list of the top-k documents. The retriever can be anything — a dense vector search, BM25, or a hybrid — because fusion, as we will see, only needs the order of each list, not the internal scores.
After this step you hold n lists, each of length k. Denote them L_1, L_2, …, L_n, where L_i ranks documents best-first for sub-query i. A document d may appear in several lists at different positions, appear in only one, or be absent from all. Its position in list i is its rank r_i(d), counting from 1 for the top result. The central question of fusion is now concrete: given that one document sits at rank 2 in one list, rank 7 in another, and nowhere in a third, how good a result is it overall?
Reciprocal Rank Fusion, the formula
RRF answers that with one line. The fused score of a document is the sum, over every list it appears in, of the reciprocal of its rank in that list, softened by a constant k:
RRF(d) = Σ_i 1 / (k + r_i(d))
r_i(d) = rank of document d in list L_i (1 = top)
k = a smoothing constant, conventionally 60
sum runs only over lists where d actually appearsThat is the entire algorithm. Rank every list, look up where each document landed, add up 1/(k+rank) across the lists, and sort documents by the total. A document at rank 1 contributes 1/(k+1); one at rank 50 contributes 1/(k+50), which is far smaller. Because the contributions are added, a document that appears in many lists — even at mediocre ranks — accumulates a high total. RRF thus rewards two things at once: being ranked highly, and being agreed upon by multiple sub-queries. Consensus is the signal.
Why ranks and not raw scores
The obvious alternative is to add up the retrievers’ similarity scores directly. RRF deliberately throws those scores away, and that is its whole point. Raw scores are not comparable across lists: a cosine similarity from a dense vector search lives on a different scale than a BM25 score, and even two runs of the same embedding model produce scores whose absolute magnitudes drift with query length and topic. Summing incomparable numbers lets whichever list has the largest raw scores dominate for no principled reason.
Rank is scale-invariant. ‘Second place’ means the same thing in every list regardless of the underlying scoring function, so RRF can fuse a vector list and a keyword list on equal footing with zero normalization or tuning. The reciprocal shape does the rest: it is steep near the top, so the gap between rank 1 and rank 2 matters a lot, and flat in the tail, so ranks 40 and 50 are treated as roughly equally weak. That matches how we actually value search results.
A worked example
Take three sub-query lists and use k = 60. Document A is at rank 1, rank 3, and rank 2 across the three lists. Document B is at rank 2 in the first list only. Document C is at rank 1 in the third list only.
A: 1/(60+1) + 1/(60+3) + 1/(60+2)
= 0.01639 + 0.01587 + 0.01613 = 0.04839
B: 1/(60+2) = 0.01613
C: 1/(60+1) = 0.01639
Fused order: A > C > BNotice what happened. Document C was ranked first by one sub-query, beating A’s best single rank — yet A wins overall, because three sub-queries independently surfaced it. A single list’s enthusiasm cannot outvote broad agreement. B, seen once at a middling rank, sinks. This is exactly the behavior we want from fusion: a document has to earn its place across perspectives, not just spike in one.
The constant k and what it tunes
The k in the denominator (conventionally 60, from the original RRF paper) is a flattener. When k is large relative to the ranks, 1/(k+r) changes slowly with r, so the difference between rank 1 and rank 5 shrinks and the number of lists a document appears in matters more than exactly where it sits. When k is small, the curve is sharp and top ranks dominate.
Concretely, at k = 60 a rank-1 hit scores 1/61 ≈ 0.0164 and a rank-2 hit 1/62 ≈ 0.0161 — nearly equal, so position is a gentle nudge and cross-list consensus dominates. Drop to k = 1 and rank 1 (0.5) is twice rank 2 (0.333), making the top of each list decisive. The default of 60 is a deliberately consensus-friendly choice, and it is robust enough that most systems never touch it; treat it as a knob you turn only if you have a measured reason to favor sharp per-list precision over agreement.