Modern RAG, Part 7: From Search Pipeline to Agentic Retrieval

So far in this series we have been building a stronger retrieval pipeline step by step.

We started with keyword search and BM25. Then we looked at vector search. Then we combined both with hybrid search and RRF. Then in the previous post I explained reranking and the difference between bi-encoders and cross-encoders. The key point was that retrieval should first find good candidates, and reranking should then select the strongest pieces of evidence.

That gives us a solid retrieval stack, but even a strong retrieval pipeline can struggle when the user asks a complex question with multiple information needs.

Real user questions are messy. They often contain multiple information needs in a single sentence.

For example:

It stops after we changed the gripper motor and shows F217, maybe calibration was missed. 
Also, do we still use motor 700-1842 in the AX-360 variant? 

That is not one retrieval task. It is several. The system must investigate the alarm. It must check the replacement context. It must search for calibration instructions. It must also check part compatibility for a specific machine variant. If we send this entire question as a single search query, we may get incomplete results.

A better approach is query planning and decomposition.

‍

‍

The system breaks down the original question into smaller sub-queries.

For example:

1. servo alarm F217 after gripper motor replacement 
Source: service tickets 

2. calibration steps after gripper motor replacement 
Source: maintenance manuals 

3. compatibility of motor 700-1842 with AX-360 
Source: parts catalog 

Each sub-query is focused.

Each sub-query can be sent to the source where the answer is most likely to be found.

That is already significantly better than searching everywhere for everything.

A service ticket system is useful for known incidents. A maintenance manual is useful for procedures. A parts catalog is useful for part numbers and compatibility. A variant table is useful for machine configurations.

Good source selection improves both quality and latency. This is where retrieval starts to become more agentic.

Agentic retrieval means the system does not just search once.

It plans. It searches. It reviews the results. It decides whether something is missing. Then it may search again.

A classic RAG pipeline looks like: query, retrieval, and generation phase. An agentic retrieval flow looks more like this:

‍

‍

That matters for real-world questions.

The first retrieval pass might find the F217 alarm description and the calibration instruction, but not the affected variants. Or it finds the part number, but not whether it is still used in the AX-360.

An agentic retriever can reflect on the retrieved evidence and ask:

Do I have enough information to answer the full question? 
Which part is still unanswered? 
Do I need another source? 
Is there a contradiction between sources? 

If something is missing, it can perform a follow-up search.

For example:

AX-360 gripper motor BOM 700-1842 replacement part

The final step is evidence consolidation.

When multiple sub-queries search across multiple sources, the system should not simply dump all passages into the LLM prompt.

It should structure the evidence.

For example:

Fault evidence: 
F217 is related to servo following error or position mismatch. 
The issue appears after gripper motor replacement in several service tickets. 

Procedure evidence: 
The maintenance manual requires recalibration after gripper motor replacement. 
The calibration procedure includes checking end positions and drive parameters. 

Compatibility evidence: 
Motor 700-1842 is listed for AX-360 in the parts catalog. 
The same motor is also used in AX-340.

This kind of structured evidence is significantly more useful than a random list of passages.

The LLM can now generate a response that is complete, grounded, and easier to verify.

‍

Conclusion

RAG should not be reduced to vector search. That is the core point of this entire series. Vector search is useful, but it is only one part of the retrieval stack.

Real users ask messy questions. They mix symptoms, assumptions, part numbers, variants, and follow-up questions in a single sentence.

The information needed to answer them can be distributed across manuals, tickets, catalogs, configuration tables, and internal documentation.

That is why strong RAG systems are built as a stack:

• ‍Keyword search provides precision for exact terms.
• ‍Vector search provides semantic recall.
• ‍Hybrid search combines both.
• ‍RRF fuses the rankings.
• ‍Reranking improves precision.
• Query planning decomposes complex questions.
• Agentic retrieval manages the entire process.

In my view, this is the direction practical RAG is heading. Not simply larger models. Not simply larger context windows.

But better retrieval systems that know what to search for, where to search, when to search again, and how to deliver the right evidence to the model.