RAG: the open-book exam

Asha runs Chai & Chapters, a café-bookshop in Bengaluru. Every day the same messages arrive. “Are you open on Monday?” “Can I return a book I got as a gift?” “Is there parking?” Every answer is in her staff handbook.

Nobody reads that handbook except Rohan, the new weekend helper. He keeps it open behind the counter, and when a customer asks something, he flips to the right page, reads it, and answers. Asha wants a chat helper on her website that works like Rohan. That is exactly what RAG is.

In the last lesson, you saw that a model cannot know Asha’s handbook by itself. It was never in its training data. Ask a plain chat app about the café’s return rule, and it will guess something that sounds like a typical shop.

A closed-book exam is answered from memory. That is a plain chat model, working from training data only. An open-book exam is answered from the book in front of you. That is RAG.

The name sounds heavy, but each word is plain.

• Retrieval: find the parts of your documents that relate to the question. Priya checking the index.
• Augmented: add those parts to the prompt, so the context window now holds the facts.
• Generation: the model writes the answer, one token at a time as you learned in module 2, but now while reading your facts.

Here is the key point. The model itself does not change. There is no new training. Asha’s handbook lives outside the model, in a searchable store. If she changes a rule today, tomorrow’s answers use the new rule.

The name comes from a 2020 research paper by a team at Facebook AI Research and university partners. Today it is one of the most common patterns behind company help desks, policy chatbots and “chat with your PDF” tools.

Let’s follow a single message through Asha’s helper. Kabir, a regular, types: “Can I return a book I got as a gift?” Watch how much happens before the model writes a single word.

Zoom out and you will see that RAG is really two jobs. One happens once, before anyone asks anything. The other happens for every single question.

The first job runs once, and again whenever the handbook changes. The second job runs for every message. How the handbook gets cut up, and how search scores meaning, is the whole of the next lesson.

RAG is not magic. It moves the problem. Now your answer is only as good as your search. A great model reading the wrong page gives a confident, well-written, wrong answer.

Imagine search had fetched the gift-cards chunk but missed the book-returns chunk. The model reads “Gift cards cannot be refunded” and tells Kabir no. Every sentence sounds right. It is still wrong, and the model followed the rules perfectly.