The second part of our series about modern Retrieval-Augmented Generation systems.
In the previous post I wrote that RAG is primarily a retrieval problem. Before the LLM can generate a useful response, the system must find the right information. The most fundamental retrieval technique is keyword search.
And although many people today primarily talk about embeddings and vector databases, keyword search is still extremely important. Especially in technical systems.
Keyword search works by matching the exact words of the query against the exact words in the documents. Documents are stored in an index. When the user searches for something, the search engine quickly finds documents that contain the same terms.
For example:
A keyword search engine can quickly find documents that contain these exact words. This is useful because technical systems often depend on exact identifiers. Error codes matter, part numbers matter, machine variants matter, software versions matter.
When the user asks for:
then a document about:
is not good enough.
The numbers look similar, but they may describe completely different problems. This is where the strength of keyword search lies. It does not try to guess the meaning. It matches what is written. One of the most common ranking algorithms for keyword search is BM25.
BM25 scores how relevant a document is for a query. It is based on three important concepts.
First: Term frequency.
If a search term appears in a document, the document might be relevant. If it appears multiple times, it might be even more relevant. But BM25 does not reward repetition endlessly. A document that repeats the word “Servo” 100 times is not automatically 100 times better.
Second: Inverse document frequency.
Rare words are more important than common words. The word “machine” may appear in almost every document. It is not very specific. But an error code like “F217” may only appear in a few documents. That makes it significantly more meaningful.
Third: Document length normalization.
Long documents naturally contain more words. Without normalization, long documents would often rank too high simply because they contain many terms. BM25 corrects for this. A short, precise service note can outperform a long, generic manual.
That is why BM25 is still useful.
It is simple, fast, and very effective when exact terms matter. In a RAG system, BM25 can be used as a strong first retrieval step. It can quickly narrow down the search space and ensure that exact identifiers are not lost. But keyword search has a limitation. Users do not always use the same words as the documents.
A user might say:
while the manual says:
A pure keyword search might miss this connection. This is where vector search becomes useful.
In the next post, we will look at the fundamental operating principles of vector search and explain why semantic similarity has become such an important part of modern RAG systems.
Richard is a member of the SABO core team, where he combines deep technical expertise with a focus on clean design and best practices. As a Senior Machine Learning Engineer and Software Architect, he works on intelligent systems such as conversational assistants and AI-driven industrial solutions. His work bridges modern software architecture with real-world machine learning applications, often involving MLOps and scalable infrastructure. In his free time, he enjoys learning new languages and exploring how they shape the way we think and communicate.
