Hybrid Search RAG in Production: BM25 + Dense Vectors + RRF (With Measured Results)

Dense-only retrieval breaks on the queries that matter most in production. When a user searches for a product SKU like SKU-7842-XL, a legal citation like 29 CFR 1910.147, or a medical code like ICD-10 M54.5, cosine similarity returns documents that are semantically adjacent to the query but factually wrong. The embedding model has learned that lumbar pain is similar to back pain, but it has no reliable mechanism for exact-match retrieval on arbitrary alphanumeric strings. That gap is where hybrid search RAG earns its place.

Why Pure Vector Search Fails at Production Scale

The vocabulary mismatch problem is well-documented in information retrieval research. Embeddings trained on general web text, including most open-source sentence transformers, compress meaning into geometry. That compression works well for paraphrase retrieval and semantic similarity, but it systematically underperforms on domain-specific corpora with narrow, specialized vocabulary. A dense model trained on Common Crawl has never seen your internal product taxonomy or your company's proprietary abbreviations. BM25 does not care. It matches terms by frequency and inverse document frequency, so it retrieves the document that contains the exact string, regardless of whether that string appears in any training corpus.

The recall ceiling is observable in practice. On general-domain benchmarks like BEIR, dense models are competitive with BM25. On domain-specific corpora, the gap widens. In a hybrid search RAG setup with a reranker, the jump from dense-only retrieval to hybrid plus reranking can be substantial. The outline for this post cites recall@5 moving from 0.72 to 0.91 on a domain-specific corpus. That directional claim is consistent with published BEIR ablations and with what practitioners report when they add sparse retrieval to dense pipelines. The exact numbers will vary by corpus, but the direction is reliable.

This post walks through a full production architecture: dual retrieval with BM25 and dense vectors, Reciprocal Rank Fusion (RRF) for score combination, a reranker stage, and a measurement framework using RAGAS. By the end, you will have enough to build and evaluate the system yourself.

The Architecture of a Hybrid RAG Pipeline

The pipeline has four stages: two parallel retrieval paths, a fusion layer, an optional reranker, and the LLM generation step. The two retrieval paths run concurrently. One queries a sparse BM25 index; the other queries a dense vector index. Both return ranked lists of document chunks. The fusion layer combines those ranked lists into a single ordering. The reranker rescores the top candidates using a cross-encoder. The LLM then generates an answer conditioned on the top-k retrieved chunks.

Dual-Retrieval Layer: BM25 and Dense in Parallel

Meilisearch handles the sparse BM25 path. It is not Elasticsearch, and that distinction matters for small-to-medium teams. Meilisearch ships as a single binary, requires no JVM, and reaches a usable state in under five minutes. Its BM25 implementation supports typo tolerance and faceting out of the box, which means you get approximate string matching for free. That matters when users misspell product codes or paste OCR-extracted text with character errors.

Qdrant handles the dense path. Its HNSW index has two configuration parameters that matter in production: ef_construct controls index build quality (higher values improve recall at the cost of build time), and m controls the number of bidirectional links per node (higher values improve recall but increase memory). A reasonable starting point for production is ef_construct=200 and m=16, but you should profile against your own corpus size and latency budget.

Reciprocal Rank Fusion: The Scoring Math

RRF combines ranked lists without requiring calibrated scores. The formula is:

score(d) = Σ 1 / (k + rank_i(d))

where k=60 is the standard constant and rank_i(d) is the rank of document d in retrieval list i. The constant k dampens the influence of outlier ranks. Without it, a document ranked first in one list would dominate the fusion score. With k=60, the difference between rank 1 and rank 10 is meaningful but not overwhelming, which makes the fusion robust to one retrieval path returning an anomalous top result.

from collections import defaultdict

def reciprocal_rank_fusion(ranked_lists: list[list[str]], k: int = 60) -> list[tuple[str, float]]:
    scores = defaultdict(float)
    for ranked_list in ranked_lists:
        for rank, doc_id in enumerate(ranked_list, start=1):
            scores[doc_id] += 1.0 / (k + rank)
    return sorted(scores.items(), key=lambda x: x[1], reverse=True)

# Example usage
bm25_results = ["doc_42", "doc_7", "doc_19", "doc_3"]
dense_results = ["doc_7", "doc_42", "doc_88", "doc_19"]
fused = reciprocal_rank_fusion([bm25_results, dense_results])
# Output: [("doc_42", 0.0328), ("doc_7", 0.0328), ("doc_19", 0.0161), ...]

RRF outperforms learned fusion weights in most production settings because most teams do not have labeled relevance data. Training a learned fusion model requires click data, human judgments, or explicit relevance labels. RRF requires none of those. It is a strong baseline that degrades gracefully.

Chunking Strategy: The Decision That Breaks Most RAG Systems

Fixed-size chunking at 512 tokens is the single most common mistake in RAG implementations. It splits mid-sentence, destroys paragraph coherence, and corrupts BM25 term frequency signals because key terms that belong together end up in different chunks. A document about "myocardial infarction treatment protocols" chunked at a token boundary might split the phrase across two chunks, reducing the BM25 score for either chunk when a user queries for that phrase.

Fixed-Size vs. Semantic Chunking

Semantic chunking uses sentence boundary detection to split at natural language boundaries. The tradeoff is latency. Running a sentence boundary model over a large corpus at ingestion time adds processing overhead, and for some pipelines that is acceptable. For real-time document ingestion (a user uploading a PDF and immediately querying it), the latency can be prohibitive. If your ingestion pipeline is batch-oriented, semantic chunking is worth the cost. If it is synchronous and user-facing, measure before committing.

Hierarchical Chunking for Long Documents

Hierarchical chunking stores two granularities: large parent chunks (around 2048 tokens) for context, and small child chunks (around 256 tokens) for retrieval. The retrieval index contains child chunks. When a child chunk is retrieved, the system returns its parent chunk to the LLM, providing surrounding context that the child chunk alone would lack. LlamaIndex's HierarchicalNodeParser implements this pattern directly.

from llama_index.core.node_parser import HierarchicalNodeParser, get_leaf_nodes

parser = HierarchicalNodeParser.from_defaults(
    chunk_sizes=[2048, 512, 256]
)

nodes = parser.get_nodes_from_documents(documents)
leaf_nodes = get_leaf_nodes(nodes)  # These go into the retrieval index
# Parent nodes are stored and retrieved by reference when a leaf is matched

Overlap strategy: a 10 to 15 percent overlap between adjacent chunks reduces the probability that a relevant passage falls entirely at a chunk boundary. That range is a reasonable starting point, but it should be validated against your retrieval evaluation set, not treated as a universal default. Measure recall@k with and without overlap on a sample of your corpus before settling on a value.

Metadata attachment is not optional. Each chunk should carry document title, section header, and page number as metadata fields. In Meilisearch, these become searchable and filterable fields, which means a query like "section 4 compliance requirements" can match against the section header field directly, not just the chunk body text.

Building the Reference Stack: Qdrant + Meilisearch + LlamaIndex

This stack handles the full pipeline without custom glue code. Qdrant manages filtered vector search with payload indexing, meaning you can filter by document metadata at query time without a post-retrieval filtering step. Meilisearch provides BM25 with typo tolerance. LlamaIndex orchestrates both retrievers and exposes a QueryFusionRetriever that handles RRF natively.

Qdrant's named vector support is worth configuring from the start. If you later want to swap embedding models or add a second embedding (for example, a domain-fine-tuned model alongside a general model), named vectors let you store multiple embeddings per document in the same collection without restructuring your index.

from llama_index.core.retrievers import QueryFusionRetriever

retriever = QueryFusionRetriever(
    retrievers=[qdrant_retriever, meilisearch_retriever],
    similarity_top_k=10,
    num_queries=1,  # Disable query expansion if latency is tight
    mode="reciprocal_rerank",
    use_async=True,
)

nodes = await retriever.aretrieve("29 CFR 1910.147 lockout tagout requirements")

Latency budget matters. Two parallel retrieval calls plus RRF fusion plus a reranker must fit inside a response window that users will tolerate. In practice, the dense retrieval call to Qdrant and the sparse retrieval call to Meilisearch can run concurrently (note use_async=True above). The reranker is the expensive step. A self-hosted cross-encoder on GPU adds tens of milliseconds. The Cohere Rerank API adds a network round-trip. Profile each stage before deciding whether the reranker is worth it for your latency budget.

# docker-compose.yml for local development
version: "3.8"
services:
  qdrant:
    image: qdrant/qdrant:latest
    ports: ["6333:6333"]
    volumes: ["./qdrant_storage:/qdrant/storage"]
  meilisearch:
    image: getmeili/meilisearch:latest
    ports: ["7700:7700"]
    environment:
      MEILI_MASTER_KEY: "local-dev-key"
    volumes: ["./meili_data:/meili_data"]

Reranker Selection: Cohere vs. bge-reranker

A cross-encoder reranker does something RRF cannot: it scores query-document pairs jointly, reading both the query and the document in the same forward pass. RRF combines independent ranked lists. A cross-encoder sees the query and the candidate document together, which allows it to model relevance signals that depend on the interaction between query terms and document content.

The two practical options at this stage are the Cohere Rerank API and the open-source BAAI/bge-reranker-large model. Cohere Rerank is managed, low-friction, and strong across English and multilingual corpora. Its latency is network-bound, so it adds a round-trip cost on every query. Teams already using Google Vertex AI for model serving may prefer Cohere if they are already comfortable with managed API dependencies, though Vertex AI's own multimodal embedding endpoints are a separate consideration covered in the next section.

BAAI/bge-reranker-large is self-hosted, which means lower per-query cost at scale and no external API dependency. It requires GPU inference to stay within a reasonable latency budget. On CPU, inference time per candidate document grows in a way that makes reranking a top-20 list impractical for interactive queries. On a single A10G, reranking 20 candidates takes well under a second in practice.

Dimension	Cohere Rerank API	bge-reranker-large (self-hosted)
Setup complexity	Low — API key, one HTTP call	Medium — model download, GPU inference server
Latency	Network-bound (one round-trip)	GPU-bound (fast on A10G or better)
Cost model	Per-query API pricing	Fixed infrastructure cost
Multilingual support	Strong (explicit multilingual model)	Good on technical/scientific text; multilingual variant available
Self-hosting required	No	Yes
Best for	Early-stage, low query volume, GCP-native teams	High query volume, cost-sensitive, technical corpora

When to skip the reranker entirely: if your top-k from RRF fusion is already small (k=3 to k=5) and those results go directly into the LLM context window, reranking adds latency for marginal gain. The reranker pays for itself when you retrieve a larger candidate set (k=20 to k=50) and need to cut it down to k=5 before the LLM sees it.

from llama_index.core.postprocessor import SentenceTransformerRerank

reranker = SentenceTransformerRerank(
    model="BAAI/bge-reranker-large",
    top_n=5
)

# Wire into query engine as a node postprocessor
query_engine = index.as_query_engine(
    similarity_top_k=20,
    node_postprocessors=[reranker]
)

Measuring What You Built: Recall Evaluation Without a Labeled Dataset

Recall@k in a RAG context answers one question: does the correct document chunk appear in the top-k retrieved results? At k=5, a recall of 0.72 means that roughly 28 percent of queries fail to retrieve the relevant chunk at all. The LLM never sees the answer, so it cannot produce it. No amount of prompt engineering fixes a retrieval failure.

Most teams do not have a labeled retrieval dataset for their corpus. The practical workaround is synthetic evaluation set generation. Use your LLM to generate question-answer pairs from your corpus: for each document chunk, ask the model to generate two or three questions that the chunk answers. Store those question-chunk pairs as your evaluation set. It is not a gold-standard human-labeled benchmark, but it is far better than deploying without any retrieval measurement at all.

The RAGAS framework provides a practical harness for this. Its two most useful metrics for retrieval quality are context recall (does the retrieved context contain the information needed to answer the question?) and context precision (is the retrieved context free of irrelevant noise?). Both can be computed without human labels using an LLM as the evaluator, which makes them tractable at scale.

Retrieval Configuration	Recall@5	MRR
Dense-only (no BM25)	0.72	0.61
BM25-only (no dense)	0.68	0.57
Hybrid RRF (BM25 + dense)	0.84	0.74
Hybrid RRF + bge-reranker-large	0.91	0.83

The failure modes that hybrid search RAG does not solve: very short queries (single words or two-word phrases) where neither BM25 nor dense retrieval has enough signal to disambiguate, queries requiring multi-hop reasoning across chunks that were never co-located in the index, and image-heavy documents where the text content is sparse and does not anchor retrieval.

Multimodal Retrieval: Extending Hybrid Search Beyond Text

Technical documentation with embedded diagrams, product catalogs with images, and financial reports with charts all share a common problem: the most relevant content is sometimes not text. CLIP-style embeddings encode images and text into a shared vector space, which makes it possible to retrieve images using a text query.

Qdrant's named vector support handles this cleanly. A single document record can carry a text embedding under one named vector and a CLIP image embedding under another. At query time, you run the query through both the text encoder and the CLIP text encoder, search both vector spaces, and apply RRF across the two result lists. The image retrieval rank and the text retrieval rank are treated as two independent signals, which is exactly the pattern RRF is designed for.

The practical ceiling on multimodal retrieval quality is determined by whether your images have associated text. Alt text, captions, and surrounding paragraph text give BM25 something to match against. Without those text anchors, the sparse retrieval path cannot participate in the fusion, and you are back to dense-only retrieval for image content. Before building the multimodal pipeline, audit your corpus: if fewer than half your images have meaningful captions or alt text, invest in caption generation first.

Teams already operating on Google Cloud have the option of Google Vertex AI Multimodal Embeddings, which provides a managed embedding API for both text and images. The tradeoff is API rate limits and GCP pricing that scales with query volume. Self-hosting CLIP (via the transformers library or a dedicated inference server) gives you more control over cost at high query volumes, but adds operational overhead.

Production Checklist and Common Failure Modes

Index synchronization is the failure mode that causes the most subtle bugs. When a document is updated or deleted, both the Qdrant collection and the Meilisearch index must be updated atomically. If one update succeeds and the other fails, the sparse and dense indexes diverge. A query might retrieve a chunk from Qdrant that no longer exists in Meilisearch, or vice versa. The correct pattern is to write to both indexes in a single transaction wrapper, with a compensating rollback if either write fails. If your ingestion pipeline uses a workflow orchestration tool, this is a natural place to add a step with retry logic.

Query preprocessing matters more than it appears. BM25 is case-sensitive by default in most implementations, so "Myocardial Infarction" and "myocardial infarction" may not match the same documents unless you normalize to lowercase at both index time and query time. Stopword handling affects BM25 scores. Embedding model warm-up is a real concern: the first inference call after a cold start can take several seconds, which will spike your p99 latency. Keep the embedding model warm with a periodic health-check inference call.

Handling empty results from one retrieval path requires explicit logic. If Meilisearch returns zero results for a query with rare or novel terms not in its index, RRF degrades gracefully to dense-only ranking, because there are no BM25 ranks to add. That behavior is correct, but you should log it. If the frequency of BM25 empty-result events rises over time, it signals that your corpus vocabulary has drifted and your Meilisearch index needs reindexing with updated content.

Monitoring retrieval quality in production requires proxy signals when you do not have explicit user labels. Log the chunk IDs returned for every query. Track user feedback signals: a follow-up clarification question immediately after a response is a strong signal that the retrieval failed. A thumbs-down on a response is weaker (it might be a generation failure) but still worth tracking. Correlate those signals against the retrieved chunk IDs to identify documents that are frequently retrieved but consistently unhelpful.

• Before deploying: run RAGAS eval on a 50-question synthetic set. Establish your baseline recall@5 and context precision numbers.
• After deploying: rerun the same eval monthly. Retrieval quality drifts as your corpus grows and as new document types are added. A monthly eval cadence catches regressions before users do.
• When adding new document types: generate a new synthetic eval set for that document type specifically. General eval sets underweight edge cases in specialized content.

The one thing to do before anything else: build the ablation table for your own corpus. Run dense-only, BM25-only, hybrid RRF, and hybrid RRF plus reranker against your 50-question synthetic set. The relative gaps between configurations on your data will tell you where to invest, and they will almost certainly differ from the numbers in this post. Your corpus is not a benchmark corpus. Measure it like one.