Hybrid Search

Hybrid search runs a lexical retriever and a vector retriever on the same query and combines their results into one ranked list. The two fail on different queries. A lexical method such as BM25 matches the exact terms in the query, so it finds an error code or a product identifier and misses a paraphrase. A vector (dense) method compares embeddings, so it finds a paraphrase and can miss a rare token that the embedding model has no concept for. Combining them lets each cover the other’s blind spot.

The combination problem

Each retriever returns a list with scores, and the scores are not comparable. BM25 scores are unbounded and depend on corpus statistics and query length, while cosine similarities lie between -1 and 1 and cluster in a narrow band. Adding raw scores lets whichever scale is larger decide. Two approaches are common.

Score normalization. Rescale each list to a common range, then combine, usually with a weighted sum. Min-max normalization maps a list’s scores to \([0,1]\) using its minimum and maximum: \( \tilde{s}=\frac{s-s_{\min}}{s_{\max}-s_{\min}} \). When every score in a list is equal, including a list with a single result, the denominator is 0, so an implementation needs a fallback such as giving all documents in that list the same constant score. It keeps score gaps, which is useful when one retriever is very confident, and it is sensitive to outliers, since one extreme score squeezes everyone else toward 0.

Rank fusion. Ignore scores and use only positions. Reciprocal rank fusion (RRF), introduced by Cormack, Clarke and Büttcher, scores a document by \[ \text{RRF}(d)=\sum_{j}\frac{1}{c+r_j(d)}, \] where \(r_j(d)\) is the document’s rank (from 1) in list \(j\) and \(c\) is a constant, often 60. A document absent from a list contributes nothing from it. Because only ranks enter, no calibration of scales is needed. A large \(c\) flattens the credit given to rank and makes RRF behave like voting on how many lists contain a document, while a small \(c\) favors the top positions.

A worked comparison

Take a hypothetical query where the lexical list scores d1=14.0, d2=9.0, d3=2.0, d4=1.5 and the vector list scores d3=0.82, d2=0.80, d1=0.55, d5=0.50.

Min-max normalization with an equal-weight sum (a document missing from a list counts 0 there) ranks d2 (1.538), d1 (1.156), d3 (1.040), then d4 and d5 (0.0). Now add one outlier to the lexical list, d6 with score 400. The lexical range stretches, and d1’s lexical contribution collapses, so its fused score falls from 1.156 to 0.188 and d3 and d6 lead. Nothing about d1’s relevance changed.

RRF on the original two lists (\(c=60\)) gives d1 0.032266, d3 0.032266, d2 0.032258, then d4 and d5 at 0.015625. The three documents that appear in both lists score almost the same, and the ones in one list score half as much. With the outlier added, d6 enters the lexical list at rank 1 and pushes the others down one place, so d1, d2 and d3 each lose about 0.0003 and d6 scores only 0.0164. The three documents found by both retrievers still lead. RRF is robust to the outlier, and it also discards the fact that BM25 was far more confident about d1 than about d3.

When it helps and when it does not

Hybrid search helps when the query set mixes exact-match needs (identifiers, codes, names, quoted phrases) with natural-language questions, which is common in technical support and documentation. Its gain is measured per query type: a hybrid that raises the average can still lower the score for a type where one retriever was right and the other was noise. It helps less when one retriever dominates the workload, in which case the second adds cost and a way to rank wrong documents highly.

Other conditions matter. Each retriever should return a deep enough list that the fused pool contains the relevant document; fusion cannot rescue a document no list ranked within its depth. Filters for permissions and dates belong inside every retriever before ranking. The lexical side depends on its analyzer: if “ERR-4029”, “ERR4029” and “err 4029” tokenize differently, exact matching silently fails.

Related terms

Hybrid search combines retrievers and happens before reranking, which reorders the fused candidates with a more expensive model such as a cross-encoder. It is not query rewriting, which changes the query, and it is not a single index that stores both sparse and dense vectors, though some products implement fusion internally.

References

• Cormack, Clarke and Büttcher, Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods — https://dl.acm.org/doi/10.1145/1571941.1572114