Retrieval-augmented generation (RAG)
Retrieval-augmented generation is a technique in which passages are retrieved from a body of data and supplied to a language model as context, so that the answer is grounded in that data rather than in the model’s training alone.
- Also called
- RAG, retrieval-augmented answering
- Good at
- Lookups that live in one passage
- Weak at
- Questions spread across a whole book
How it works
The pipeline is: cut the source into fragments, embed them, embed the question, retrieve the fragments that score highest, and generate an answer from those.
That is an excellent fit for lookup. “What is the refund window in this contract?” has an answer that lives in one paragraph, and retrieval will find that paragraph.
Where it stops working
It degrades in a specific and predictable way as the question stops being a lookup:
- Evidence spread across a document. Where does the author change position? No single fragment contains the change; it is visible only across chapters. Retrieval returns the fragments most similar to the question, which is not the same as the fragments that together answer it.
- Absence. What does this paper explicitly not claim? There is nothing to embed. A search for what is not there returns what is.
- Disagreement between sources. Retrieval optimises for similarity to the question, so it will happily return five passages that agree and none that dissent — and a smooth summary of five agreeing passages reads exactly like a consensus.
- Structure. Chunking destroys the thing that made a book navigable. A pile of paragraph-sized fragments has no chapters, no argument order, no “earlier” and “later”.
The question needs chapters 4 and 11. Retrieval returns what looks most like the question — and the answer gives no sign of what it never saw.
The uncomfortable part is that none of these failures announce themselves. A retrieval miss and a retrieval hit produce output of identical confidence, and nothing in the answer says that three relevant passages were never shown to the model.
Retrieval quality can be improved — hybrid keyword-plus-semantic matching, reranking, larger overlapping windows — and all of it helps. None of it changes the fact that the model is answering from a sample it did not choose and you cannot see.
How MindMax handles it
By not making retrieval the default. If a document fits inside a stated reading budget, its full text goes into context and nothing is retrieved at all — see full-text-first answering. No embedding pass runs, no vector store is built, and there is no indexing wait between opening a folder and asking it something.
For material genuinely too large, structure is used before similarity: the chapter map is read first, then the sections that bear on the question are read in full — the way a person uses a table of contents — rather than sampled as fragments that scored well. Looking for one specific sentence is treated as a lookup and done with keyword search, because exact terms are what similarity blurs.
This is what is often called vectorless RAG: the model decides what to read by reasoning over structure, instead of a similarity score deciding for it.
Search still exists underneath, because a thousand documents will never fit in a context window. What changes is that it locates what to read rather than standing in for the reading, and that every claim it produces carries a quote you can check in one click.
The document you added is read first; the rest of the folder is searched only when the question needs it.
Common questions
Why does RAG miss information in long documents?
Because it answers only from the fragments most similar to the question. When the evidence is spread across chapters, is an absence, or is a disagreement between sources, the fragments that matter are often not the most similar ones — and the answer gives no sign that they were left out.
Is RAG still needed now that context windows are large?
For large collections, yes: thousands of documents will not fit in any window. For a single paper or report, often not — a document that fits a sensible reading budget can simply be read whole, which removes retrieval misses entirely.
Does MindMax use RAG?
Not as the default. A document that fits the reading budget is read whole; a longer one is navigated by its chapter map and read section by section. Search is used to locate documents and sections to read, and every claim carries a quote checked against the source.