Two retrievers, two blind spots

Start from what each retriever actually computes, because that is where their complementarity comes from. A dense retriever encodes the query and every document into vectors in R^d and ranks by similarity, typically cos(q, d) = (q · d) / (||q|| ||d||). It matches on meaning: ‘heart attack’ and ‘myocardial infarction’ land close together even with zero shared words. A sparse retriever like BM25 scores on term overlap, weighting each shared term by how often it appears in the document and how rare it is in the corpus. It matches on exact lexical signal: a product code like SKU-4417X, a surname, a function name.

Their blind spots are almost mirror images. Dense embeddings blur rare, out-of-vocabulary, and precise tokens — the exact string gets averaged into a smooth semantic neighborhood, so SKU-4417X retrieves ‘similar-looking part numbers.’ Sparse retrieval is literal: it cannot bridge a paraphrase, so a query and its answer that share no vocabulary score zero. The dense and sparse siblings in this series cover each mechanism in depth; here they are the two inputs to fuse.

Advertisement

Why complementary beats better

The case for hybrid is statistical, not just anecdotal. If two retrievers made the same mistakes, combining them would buy nothing — you would just re-rank the same errors. Hybrid works because their errors are decorrelated: the queries where dense fails (exact codes, names, negations) are largely disjoint from the queries where sparse fails (synonyms, paraphrase, cross-lingual meaning). When two systems each recall roughly 70% of the relevant documents but miss different ones, their union recalls far more than either — the same reason an ensemble of decorrelated classifiers beats its members.

This reframes the goal. You are not chasing the single best retriever; you want two whose strengths cover each other’s gaps. A mediocre lexical retriever paired with a strong dense one often beats a slightly stronger dense one alone, because it contributes the exact-match recall the dense model structurally cannot. The fusion step converts two overlapping-but-different candidate sets into one ranked list — and how you fuse decides whether you keep that gain or throw it away.

Advertisement

The fusion problem: two lists, one ranking

Concretely, each retriever returns its top-k as a ranked list of (document, score) pairs. Dense might return [(D, 0.83), (A, 0.81), (B, 0.79), …]; sparse might return [(C, 14.2), (A, 11.6), (E, 9.4), …]. You need one merged, de-duplicated ranking. Two obstacles make this non-trivial.

Incompatible scales. Cosine similarity is bounded, often in [-1, 1] and in practice clustered in a narrow band near the top; BM25 is unbounded above and its magnitude depends on query length, term rarity, and corpus statistics. The number 0.81 and the number 11.6 live in different universes — adding them lets BM25 silently dominate. Partial overlap. A document can appear in one list, both, or neither, and a document ranked #1 by dense may not appear in the sparse top-k at all, so its sparse score is not just small — it is missing. Any fusion rule has to define what an absent document contributes. The two mainstream answers differ in exactly how they handle these two problems.

Reciprocal Rank Fusion: fuse on rank, not score

Reciprocal Rank Fusion (RRF) sidesteps the scale problem entirely by discarding the scores and keeping only the rank each list assigns. For a document d, its fused score sums a small reciprocal over every retriever r that ranked it:

RRF(d) = Σ_r  1 / (k + rank_r(d))

  rank_r(d) = position of d in list r (1 = top)
  k         = smoothing constant, conventionally 60
  d absent from list r  →  that term is simply 0

The design is clever in three ways. Because only rank enters, a bounded cosine and an unbounded BM25 become directly comparable — no calibration, no normalization, no per-corpus tuning. The reciprocal makes the contribution steeply top-heavy: rank 1 contributes 1/61 ≈ 0.0164, rank 2 1/62 ≈ 0.0161, but rank 100 only 1/160 ≈ 0.0063 — so being near the top of either list matters and the long tail barely moves the result. The constant k controls how sharply: small k exaggerates the gap between rank 1 and rank 2; large k flattens the curve so deeper ranks still count. The real power is consensus — a document ranked decently by both retrievers accumulates two terms and outranks one ranked #1 by only a single retriever.

A worked RRF example

Take two lists over documents {A, B, C, D, E} with the conventional k = 60:

Dense  ranks:  A=1,  B=2,  C=3,  D=4
Sparse ranks:  C=1,  E=2,  A=3,  B=4

RRF(A) = 1/(60+1) + 1/(60+3) = 0.01639 + 0.01587 = 0.03226
RRF(C) = 1/(60+3) + 1/(60+1) = 0.01587 + 0.01639 = 0.03226
RRF(B) = 1/(60+2) + 1/(60+4) = 0.01613 + 0.01563 = 0.03176
RRF(E) = 1/(60+2)            = 0.01613
RRF(D) = 1/(60+4)            = 0.01563

Fused order:  A ≈ C  >  B  >  E  >  D

Read what the arithmetic did. A and C rise to the top not because either topped a single list, but because each was ranked well by both retrievers — A was dense-#1 and sparse-#3, C was sparse-#1 and dense-#3, and their two terms sum to identical consensus scores. B also appears in both lists but lower in each, so it trails. D and E, each seen by only one retriever, sink — a lone rank-2 (E) or rank-4 (D) cannot out-accumulate two mediocre ranks. That is exactly the behavior you want: agreement across independent signals is promoted, single-system enthusiasm is discounted.

Weighted score combination: keep the scores

RRF throws information away — it knows A was dense-#1 but not that A’s cosine (0.81) was almost identical to B’s (0.79), a gap RRF treats as a full rank step. Weighted combination keeps the raw scores and blends them with a mixing weight, usually written as a convex combination:

score(d) = α · dense(d) + (1 - α) · sparse(d)

  α ∈ [0, 1]   mixing weight
  α = 1  → pure dense    α = 0  → pure sparse

Written that way it looks trivial, and that is the trap. Plug in the raw numbers — dense(A) = 0.81, sparse(A) = 11.6 — and with α = 0.5 you get 0.5×0.81 + 0.5×11.6 = 6.2, a figure almost entirely determined by the BM25 term. The normalization problem is fundamental here: adding a quantity in [0, 1] to one in [0, 30] is not a weighted blend, it is BM25 wearing a token dense coefficient. Before any α is meaningful, the two score distributions must be mapped onto a common scale.