The fifth part of our series about modern Retrieval-Augmented Generation systems.
In the previous post I explained why vector search alone is not enough. It can find semantically related documents, but might blur important technical details like alarm codes, part numbers, or software versions.
That is why many practical RAG systems use hybrid search. Hybrid search combines two different retrieval methods.
Keyword search finds exact terms. Vector search finds semantic similarity. This is a strong combination for RAG systems, as real user questions often require both.
For example:
Keyword search helps retrieve exact identifiers like:
Vector search helps find related phrasing like:
But when we have two result lists, we need to merge them. That is not as straightforward as it sounds. Keyword search and vector search produce different types of scores. BM25 produces one type of score. Cosine similarity produces another. These scores are not directly comparable. A BM25 score of 12 does not mean the same as a vector similarity score of 0.82.
If we merge the results naively, one method could dominate the other. This is exactly where Reciprocal Rank Fusion, or RRF for short, becomes useful.
RRF does not care about the original scores. It only cares about the ranks. If a document appears near the top of a ranking, it receives a boost. If it appears near the top of multiple rankings, it receives an even better score.
The idea is simple. A document that ranks well in both keyword search and vector search is likely important.
For example, imagine keyword search returns document A, B, C and vector search returns document C, A, D.
RRF combines these rankings by looking at where each document appeared.
Document A ranks well in both lists and therefore receives a strong combined score.
Document C ranks first in vector search and third in keyword search, so it also receives a good score.
Document D only appears in vector search and at a lower rank, so it receives a weaker score.
The advantage of RRF is its robustness. It does not need to normalize BM25 scores and vector scores to the same scale. It simply uses the rank positions. That makes it a practical and simple method for combining multiple retrieval methods.
In modern RAG systems, a typical flow looks like this:
This already gives us a much better retrieval pipeline than vector search alone. But we still have one limitation. RRF does not read the query and the document deeply together. It only knows where a document appeared in the previous rankings.
It does not ask:
That is why many systems add one more step after hybrid search.
Reranking.
In summary: Hybrid search gives us a significantly better candidate list than keyword search or vector search alone. RRF is a simple and robust method for merging different rankings without directly comparing incompatible scores.
But there is still one important limitation: RRF does not read the query and the document deeply together. In the next post, I will explain reranking and show why cross-encoders are often used to improve retrieval precision.
Richard is a member of the SABO core team, where he combines deep technical expertise with a focus on clean design and best practices. As a Senior Machine Learning Engineer and Software Architect, he works on intelligent systems such as conversational assistants and AI-driven industrial solutions. His work bridges modern software architecture with real-world machine learning applications, often involving MLOps and scalable infrastructure. In his free time, he enjoys learning new languages and exploring how they shape the way we think and communicate.
