Hybrid RAG: Fusing Keyword and Vector Search with Reranking

Hybrid RAG runs two retrievers over the same query and fuses their results: a sparse keyword index (BM25) and a dense vector index. Each catches what the other is blind to — BM25 nails the exact error code or product SKU a vector embedding smears away, and the vector finds the paraphrase that shares no words with the document. Fusing them, then reranking the shortlist, is the single most reliable retrieval-quality upgrade in production.

Dense retrieval has a precision hole

Vector search is built for meaning, which is exactly why it fumbles specifics. "Error TX-409" and "Error TX-410" sit almost on top of each other in embedding space, and a rare identifier the embedding never really learned lands somewhere vague. Keyword search has the opposite profile: it's exact to a fault and goes blind the moment the user paraphrases. Neither is wrong — they fail in different directions, which is the whole reason to run both.

Reciprocal Rank Fusion needs no shared scale

The trap is trying to add a BM25 score to a cosine similarity — they live on incompatible scales and the blend is meaningless. Reciprocal Rank Fusion sidesteps it entirely by throwing away the scores and keeping only the ranks: each document scores 1 / (k + rank) in each list, and you sum across lists. A document both retrievers rank highly wins; a document only one finds still places. No tuning of relative weights, no normalization — just position.

// two retrievers in parallel, fused by rank, then reranked
const [sparse, dense] = await Promise.all([
  bm25.search(q, 50),            // exact terms, codes, names
  vectorIndex.search(embed(q), 50), // meaning, paraphrase
]);

const fused = rrf([sparse, dense], 60); // score = Σ 1/(60 + rank)
const top = await reranker.rank(q, fused.slice(0, 20)); // cross-encoder
return top.slice(0, 5);          // small, high-precision context

The cross-encoder earns its cost on a small top-k

Retrieval is recall-first: cast a wide net, accept some noise. Reranking is precision-last: a cross-encoder reads the query and each candidate together — not as two pre-computed vectors — and scores true relevance. It's too slow to run over the whole corpus, which is the point: you only ever run it on the ~20 survivors of fusion, so you pay for accuracy exactly where it changes the answer. Fewer, better chunks also means a tighter context window and a cheaper generation.

One retriever is a bet that meaning or keywords matter more. Hybrid RAG refuses the bet — retrieve both ways, fuse by rank, and let a cross-encoder spend its compute on the handful of chunks that actually reach the prompt.

Hybrid retrieval builds directly on vector foundations and only pays off when your chunking strategy gives each retriever clean units to rank — the same pipeline that grounds the streamerOS support agent. Continue on the roadmap.