Standard RAG vs. Agentic RAG: Moving Retrieval From Pipeline Stage to Runtime Decision

The assumption every RAG demo makes Standard RAG assumes the user's question maps onto one vector search. One query in, one embedding, one top-k lookup, one answer. That assumption holds up in demos, because demos ask demo questions. "What's our parental leave policy?" is one document. Retrieve it, stuff it into the prompt, done. Then you ship, and a real user types: "Did the carrier rate change we approved in Q2 actually reduce our cost per shipment in the Northeast, and does that hold if I exclude the Boston depot?" That question needs a policy document, a rate table, a transactional aggregate, and a filtered re-computation. Your retriever will embed the whole sentence, find the three chunks nearest to it in vector space, and hand the model text that is topically adjacent and factually useless. The model, being a good sport, will answer anyway. The problem isn't the embedding model or the chunk size. You hardcoded how many times to retrieve, and where to retrieve from, at design time, for a question you hadn't read yet. Agentic RAG moves that decision to runtime. Planners, memory, MCP servers, sub-agents: all of it is implementation detail hanging off that one change. Architecture 1: standard RAG is a straight line STANDARD RAG — fixed pipeline, one pass ┌──────┐ 1. prompt+query ┌─────────────┐ │ User │ ───────────────────► │ Chat UI │ └──────┘ └──────┬──────┘ ▲ │ 2. query │ 6. response ▼ │ ┌─────────────┐ │ │ Retriever │ │ └──────┬──────┘ │ │ 3. fetch (top-k, one shot) │ ▼ │ ┌───────────────────────────┐ │ │ Knowledge Sources │ │ │ docs · PDFs · code · DB │ │ │ APIs · web index │ │ └───────────┬───────────────┘ │ │ 4. chunks │ ┌──────▼──────┐ └──────────────────────────│ LLM │ └─────────────┘ 5. prompt + query + enhanced context The defining property is that the model is never consulted about retrieval. It receives context and produces text, and retrieval already finished by the time it runs. That's a design choice with real advantages. One embedding call plus on