Hybrid Search

Dense vector search and classic keyword search fail in opposite ways. Vector embeddings capture synonyms and conceptual intent ('how do I reset my credentials'), but smear out exact tokens such as part numbers (`SKU-9941-B`), statutory references (`Section 75 Consumer Credit Act`), acronyms, and stack-trace error codes. Conversely, sparse lexical algorithms like BM25 score exact term frequency and inverse document frequency with pinpoint precision, but miss relevant documents that use different phrasing.

A hybrid search pipeline executes both queries in parallel against the corpus. The lexical leg scores sparse token matches while the vector leg retrieves top semantic neighbours using ANN search. A fusion algorithm, most commonly Reciprocal Rank Fusion (RRF) or weighted score normalisation, merges the two candidate lists into a unified ranking. In production RAG pipelines, the top 20 to 50 fused results are then passed to a cross-encoder reranker model, which inspects the full query-passage pairs together to order the final five chunks sent to the LLM.

For example, a UK engineering parts distributor implemented hybrid search + reranking across 800,000 technical datasheets, lifting top-5 retrieval recall from 68 per cent (vector-only) to 91 per cent across mixed natural-language and part-number queries.

Switching from naive vector-only search to hybrid search with a cross-encoder reranker is consistently the highest-ROI architectural upgrade in enterprise RAG. It eliminates the frustrating failure mode where a user pastes an exact policy code, postcode, or customer reference into an AI assistant and gets back vaguely related boilerplate instead of the exact record.

The trade-off is a modest increase in storage footprint (maintaining both an inverted lexical index and an HNSW vector index) and 50 to 150 milliseconds of added latency if a neural reranker stage is included. Because almost every major search store (Postgres with `pgvector` + full-text search, Elasticsearch, Qdrant, Weaviate, and Pinecone) now supports hybrid retrieval natively, it should be the default baseline for production RAG.