Lesson 10.4 · 25 min

Hybrid Search: Combining Sparse and Dense Retrieval

Keyword search finds "E1042" but misses "my printer won't print bills"; semantic search does the opposite. What if we did not have to choose?

In short: Hybrid search runs keyword search (usually BM25) and semantic (vector) search on the same query and merges the two ranked lists into one. The most popular merge is Reciprocal Rank Fusion, which adds up 1 / (k + rank) from each list and ignores raw scores; the alternative is to normalise both score scales and blend them with a weight. The result catches both exact terms and paraphrases.

Keyword search, and why it is not enough alone

Our running example: the support search for an office printer company. Customers type things like "E1042 invoice printing fails" or "my printer won't print bills".

Keyword search (lexical search) finds documents that contain the query's words. It uses an inverted index (word → list of documents) and ranks with BM25, a formula that rewards documents where query words appear often (term frequency), especially rare words (inverse document frequency, so "E1042" counts far more than "the"), and slightly penalises very long documents.

Strengths: exact matching of codes, names and jargon; fast; no model needed; easy to explain. Weakness: it only matches spelling. "Won't print bills" shares no word with "invoice printing problems", so keyword search ranks that article poorly or not at all.

Think of it like two detectives One detective checks fingerprints: exact, never confused, but useless if the suspect wore gloves. The other reads behaviour and motive: good with vague clues, but can be fooled by a lookalike. A smart chief listens to both and trusts the suspects both detectives point at.

Semantic search, and why it is not enough alone

Semantic search embeds the query and every document into vectors with an embedding model and returns the documents whose vectors are closest (usually by cosine similarity). It understands that "won't print bills" means "invoice printing fails". It handles synonyms, paraphrases and even other languages.

Its weakness is the mirror image. An embedding summarises general meaning, so specific rare tokens can get blurred. "E1042" might embed close to every other error-code article, and the exact one may not come first. Product SKUs, people's names, version numbers and new jargon the model never saw in training are common trouble spots. Semantic search also struggles to explain why something matched.

Pause and think: Which search would you trust more for the query "firmware 4.2.17 release notes", and why?

Keyword search. The important part is the exact version string 4.2.17, which BM25 matches exactly, while an embedding might treat all firmware release notes as nearly the same meaning.

What is hybrid search, and how it runs both searches

Hybrid search runs both a keyword search and a semantic search for the same query and combines their results into one ranked list. Documents that score well in either list can surface, and documents that score well in both rise to the top.

The two retrievers are independent, so they can run in parallel, and total latency is roughly the slower of the two plus a tiny merge step. Each retriever usually returns more candidates than we finally need (say 50 each for a final 10), so that good documents from one side are not cut too early.

Many systems support this natively: Elasticsearch and OpenSearch, Weaviate, Qdrant, Milvus, Vespa, Pinecone (with sparse-dense vectors) and PostgreSQL with full-text search plus pgvector. The API names and defaults differ, so check the docs of the system you use.

How the two result lists are combined

Merging is harder than it looks because the two scores live on different scales. BM25 scores are unbounded: 1.8 might be high for one query and low for another. Cosine similarities sit roughly between 0 and 1, and for many models almost everything lands between 0.3 and 0.9. Adding them directly would let whichever scale is larger dominate. There are two standard fixes: ignore the scores and use ranks (RRF), or normalise the scores onto a common scale before blending.

Reciprocal Rank Fusion (RRF)

Reciprocal Rank Fusion (Cormack, Clarke and Büttcher, 2009) gives each document a score based only on its position in each list. Rank 1 earns the most, and the reward shrinks slowly as rank grows. A document missing from a list gets nothing from that list.

Small numbers: BM25 ranks our documents C, A, D, B. Vector search ranks them C, B, A, D. Document A is 2nd and 3rd: 1/62 + 1/63 = 0.01613 + 0.01587 = 0.0320. Document B is 4th and 2nd: 1/64 + 1/62 = 0.01563 + 0.01613 = 0.0318. So A edges out B, because it was decent in both lists, while B was great in one and last in the other. C is first in both and wins with 2/61 ≈ 0.0328.

RRF step by step

1. Get both ranked lists: Run BM25 and vector search; keep each list's order, throw away the scores.
2. Score each appearance: For every document in every list, compute 1 / (60 + rank).
3. Add up per document: Sum the contributions from both lists. Missing from a list = 0 from that list.
4. Sort: Order documents by the summed score and return the top k.

Why is RRF so popular? It needs no score calibration, has essentially one knob (k), is robust across queries, and works for any number of lists. Its downside: it throws away how much better the first result was than the second. If BM25 found one exact match with a huge score, RRF only knows "it was rank 1".

Pause and think: With k = 60, how much does being rank 1 vs rank 10 in one list change a document's RRF score?

1/61 ≈ 0.0164 vs 1/70 ≈ 0.0143, a difference of only about 0.002. The large k flattens the curve, so agreement between lists matters more than a top spot in just one.

Weighted score combination and normalisation

The other approach keeps the scores but first puts both on a 0-to-1 scale. The simplest method is min-max normalisation: in each list, the best score becomes 1, the worst becomes 0, and the rest are scaled in between. Then we blend with a weight α (alpha):

Example: BM25 scores are C = 1.78, A = 1.42, D = 0.78, B = 0. Min-max gives C = 1, A ≈ 0.80, D ≈ 0.44, B = 0. Vector scores C = 0.83, B = 0.62, A = 0.58, D = 0.40 become C = 1, B ≈ 0.51, A ≈ 0.42, D = 0. With α = 0.5, A gets 0.5·0.42 + 0.5·0.80 ≈ 0.61. Other options include z-score normalisation (subtract the mean, divide by the standard deviation) or using known theoretical score ranges.

Weighted blending keeps score strength and gives an intuitive dial, but min-max is sensitive to outliers and to how many results each list returns, and the best α varies by dataset. Tune α on a set of real queries with known good answers; values around 0.5 to 0.7 toward vector are a common starting point, but there is no universal best value.

Code you can run: BM25 + vectors + RRF

This script scores four printer-support documents with a real BM25 implementation, uses made-up cosine scores for the vector side (we have no embedding model here), then fuses them with RRF and with a weighted blend.

hybrid_search.py

import math
docs = {"A": "error E1042 when printing invoices",
"B": "printer shows a paper jam warning",
"C": "how to fix invoice printing problems",
"D": "E1042 firmware update notes"}
query = "E1042 invoice printing fails"
def bm25(query, docs, k1=1.5, b=0.75):          # classic keyword scoring
toks = {i: d.lower().split() for i, d in docs.items()}
avg = sum(len(t) for t in toks.values()) / len(toks)
out = {}
for i, t in toks.items():
s = 0.0
for w in query.lower().split():
n = sum(w in x for x in toks.values())       # docs containing w
if n == 0: continue
idf = math.log(1 + (len(toks) - n + 0.5) / (n + 0.5))
tf = t.count(w)
s += idf * tf * (k1 + 1) / (tf + k1 * (1 - b + b * len(t) / avg))
out[i] = s
return out
kw = bm25(query, docs)
vec = {"A": 0.58, "B": 0.62, "C": 0.83, "D": 0.40}  # pretend cosine scores
kw_rank = sorted(kw, key=kw.get, reverse=True)
vec_rank = sorted(vec, key=vec.get, reverse=True)
print("BM25   :", {d: round(kw[d], 2) for d in kw_rank})
print("vector :", vec_rank)
# Reciprocal Rank Fusion: only ranks matter, k = 60 by convention
rrf = {d: 1 / (60 + kw_rank.index(d) + 1) + 1 / (60 + vec_rank.index(d) + 1) for d in docs}
print("RRF    :", {d: round(s, 4) for d, s in sorted(rrf.items(), key=lambda x: -x[1])})
# Weighted fusion: min-max normalise each list to 0..1 first, then blend
def norm(s):
lo, hi = min(s.values()), max(s.values())
return {d: (v - lo) / (hi - lo) for d, v in s.items()}
nk, nv, alpha = norm(kw), norm(vec), 0.5
mix = {d: alpha * nv[d] + (1 - alpha) * nk[d] for d in docs}
print("alpha=.5:", {d: round(s, 2) for d, s in sorted(mix.items(), key=lambda x: -x[1])})

Output:

BM25   : {'C': 1.78, 'A': 1.42, 'D': 0.78, 'B': 0.0}
vector : ['C', 'B', 'A', 'D']
RRF    : {'C': 0.0328, 'A': 0.032, 'B': 0.0318, 'D': 0.0315}
alpha=.5: {'C': 1.0, 'A': 0.61, 'B': 0.26, 'D': 0.22}

Notice document B ("paper jam warning"): vector search ranks it 2nd because it is about printers, but it has nothing to do with E1042 or invoices, and BM25 gives it 0. Both fusion methods push it below A, the article that actually mentions the error code. That is hybrid search doing its job. Also note "invoices" in A does not match "invoice" in the query: real keyword engines apply stemming to fix that.

Choosing a fusion method, and hybrid search in the real world

Hybrid search in practice E-commerce search mixes exact SKU and brand matches with descriptive queries like "warm waterproof jacket for kids". Enterprise RAG over manuals and tickets combines error codes with natural questions. Legal and medical search needs exact terms (statute numbers, drug names) plus concept matching. A common modern recipe is: hybrid retrieval of about 50–100 candidates, then a reranker to pick the final top 5–10.

Common mistakes Adding raw BM25 and cosine scores without normalising (one scale silently dominates). Returning only the top 5 from each retriever before fusing, so good candidates are cut early. Tuning α on a handful of queries. Forgetting that the keyword side needs proper text processing (lowercasing, stemming, language analysers). Letting the two indexes drift out of sync when documents are updated or deleted.

Common mistakes and how to spot them

Fusion bugs raise no error. The results just get quietly worse. Two of them are easy to see once we put small numbers on them.

Mistake 1: one outlier flattens min-max. A query contains an exact error code, so one document gets a huge BM25 score and the rest get ordinary ones (illustrative scores below).

Q, R and S are squeezed into the range 0 to 0.04. In the blend, the keyword side now has almost no say in how those three are ordered; the vector side decides alone. That is fine when P truly is the answer. It hurts when P is a document that merely repeats a rare word. To spot it, print the normalised scores for a few queries: if all but one sit near 0, this is the cause. RRF does not have the problem, because it never looks at score sizes.

Mistake 2: cutting candidates too early. Suppose each retriever returns only its top 5, and the right document is 8th in both lists. It never reaches the fusion step. With 50 candidates each it would score 2 / (60 + 8) ≈ 0.0294, well above a document that is first in one list only (1 / 61 ≈ 0.0164).

A quick fusion health check

1. Collect cases: Take about 20 real queries for which we know the right document.
2. Record three ranks: For each query, note the rank of the right document in the BM25 list, in the vector list and in the fused list.
3. Absent from both lists: Then fusion is innocent. The problem is retrieval itself or the candidate depth.
4. Present in one list only: This is the case hybrid search exists for. Check that the fused rank is still good enough to reach the final top k.
5. Fused rank worse than both lists: This usually points to the fusion step: unnormalised scores, an outlier, or a weight far from sensible.

Practice: try it yourself

We will write RRF as a small reusable function that takes any number of ranked lists, an adjustable k and optional weights per list. Then we feed it two lists that mostly disagree and watch how k decides between "agreement" and "a top spot".

practice_rrf.py

# Reciprocal Rank Fusion for any number of ranked lists.
def rrf(lists, k=60, weights=None):
weights = weights or [1.0] * len(lists)
scores = {}
for w, ranking in zip(weights, lists):
for rank, doc in enumerate(ranking, start=1):
scores[doc] = scores.get(doc, 0.0) + w / (k + rank)
return sorted(scores.items(), key=lambda item: -item[1])
# Two retrievers that mostly disagree. Only doc D appears in both lists.
bm25_list   = ["A", "B", "C", "D"]      # A = exact error-code match
vector_list = ["E", "F", "G", "D"]      # E = best paraphrase match
def show(label, fused, top=3):
print(f"{label:14}", [(doc, round(score, 4)) for doc, score in fused[:top]])
show("k=60", rrf([bm25_list, vector_list], k=60))
show("k=1", rrf([bm25_list, vector_list], k=1))
show("k=60, bm25 x2", rrf([bm25_list, vector_list], k=60, weights=[2.0, 1.0]))
# A document missing from one list simply gets nothing from that list.
a_score = dict(rrf([bm25_list, vector_list]))["A"]
print(f"A appears once: {a_score:.4f} = 1/61 = {1 / 61:.4f}")

Output:

k=60           [('D', 0.0312), ('A', 0.0164), ('E', 0.0164)]
k=1            [('A', 0.5), ('E', 0.5), ('D', 0.4)]
k=60, bm25 x2  [('D', 0.0469), ('A', 0.0328), ('B', 0.0323)]
A appears once: 0.0164 = 1/61 = 0.0164

Now change it:

• Move D to second place in the vector list: ["E", "D", "F", "G"]. Work out the new k = 60 score of D by hand (1/64 + 1/62) before you run it.
• Add a third list, ["D", "A"], as if a title-match retriever had run too, and pass all three lists to rrf. Predict which document is first at k = 1 now.
• Call rrf with k=0. Predict the scores of A and D, and say why a k of 0 makes a single first place count for so much.

Pause and think: With k = 60, D wins although it is last in both lists. With k = 1, A and E beat it. What does k really control?

How steeply the reward falls with rank. With a small k, first place is worth far more than fourth (1/2 against 1/5), so one top spot beats agreement. With a large k the curve is nearly flat (1/61 against 1/64), so appearing in two lists is worth almost double and agreement wins.

Pause and think: With the BM25 list weighted 2, document B (second in BM25 only) is ranked above E (first in vector search). Is a weight of 2 a gentle nudge?

No. At k = 60 all ranks earn nearly the same amount, so doubling one list makes every position in it worth more than first place in the other: 2/62 ≈ 0.0323 against 1/61 ≈ 0.0164. Weights in RRF are strong; small changes such as 1.2 are usually enough.

Key takeaways

• Keyword search (BM25) nails exact terms; semantic search handles meaning; each covers the other's blind spot.
• Hybrid search runs both retrievers, usually in parallel, and fuses their ranked lists.
• RRF scores each document as ∑ 1 / (k + rank), with k = 60 by default, and needs no score calibration.
• Weighted fusion normalises scores (e.g. min-max) and blends them with α; it keeps score gaps but needs tuning.
• A strong production recipe: hybrid retrieval of many candidates, then a reranker for the final top results.

Key terms

• BM25: A keyword ranking formula based on term frequency, inverse document frequency and document length.
• Hybrid search: Running keyword and vector search together and merging their results.
• Reciprocal Rank Fusion: Merging ranked lists by summing 1 / (k + rank) for each document.
• Min-max normalisation: Rescaling scores so the lowest becomes 0 and the highest becomes 1.
• Alpha (α): The weight that balances vector and keyword scores in a weighted hybrid blend.
• Stemming: Reducing words to a root form so "invoices" matches "invoice".

← 10.3 Semantic Search: Finding Meaning, Not Just Keywords · 10.5 Rerankers: Re-Scoring Retrieved Results by Relevance →