Why AI makes things up (hallucinations)

Asha is finishing her college project in Bhopal at midnight. She asks an AI for three books on water conservation in India. It gives her three neat references: authors, titles, publishers, years. The next morning, the librarian searches every catalogue. Two of the books do not exist. The authors sound real. The titles sound real. They were made up, word by fluent word.

Nobody programmed the AI to lie. So where did those books come from? The answer follows straight from everything you have learned so far.

When an AI states something false as if it were true, people call it a hallucination. The word is a bit dramatic. The model is not seeing things. It is producing text that fits the pattern of a good answer, without anything inside it checking whether the content is real.

• Made-up facts: a wrong date, a wrong capital, an award someone never won.
• Fake references: books, research papers, court cases or web links that do not exist.
• Invented quotes: words put in a famous person’s mouth.
• Wrong numbers: sums, percentages and prices that look exact but are not.
• Unfaithful summaries: a summary that adds or changes things the document never said.

Remember how the model was built. It learned by predicting the next token across a huge library. Then it was fine-tuned to always respond helpfully, and rated by people who liked clear, confident answers. Nowhere in that recipe is a step that says “look this up and check it”.

A model does not store facts like a database stores rows. It stores patterns in its numbers. A fact seen thousands of times, like “Jaipur is the capital of Rajasthan”, becomes a strong pattern. A fact seen once or twice becomes a faint one. A fact it never saw is not there at all. But in every case, the model still knows what an answer usually looks like.

So when Asha asked for books, the model knew the shape of a reference perfectly. It gave an Indian-sounding author, a title with “water” and “India” in it, a real publisher and a believable year. It filled in each slot with the most likely tokens. The result looked exactly like a real reference, because looking like one is what it was trained to do.

Researchers have also pointed out a quieter cause. In training and in many tests, a guess can score points while “I don’t know” scores zero. Like a student in an exam with no negative marking, the model learns that guessing pays. Labs now try to reward honest uncertainty, but the habit runs deep.

That last trigger is sneaky. Sachin Tendulkar’s highest Test score is 248 not out. Ask a model why he scored a triple century, and a weaker one may cheerfully explain the innings that never happened. The question itself pulled the answer off track.

Hallucinations are hard to catch because they sit right next to true facts, in the same calm tone. Train your eye with a few real-looking examples.

You cannot remove hallucinations completely. You can make them much rarer and much quicker to catch. The biggest fix is to stop asking the model to remember, and start asking it to read.

1. 1

   Give it the facts

   Paste the document, policy or notes into the prompt and say “answer only from this text”. Module 4 shows how apps do this at scale with RAG.

2. 2

   Allow “I don’t know”

   Say it in plain words: “If the answer is not in the text, say you don’t know.” Models follow this far better when it is stated.

3. 3

   Ask where it came from

   Request a quote or source for each claim. Then open the source. A citation you have not checked is still a claim.

4. 4

   Use tools for exact work

   Let a search tool fetch fresh facts and a calculator do the maths. Module 5 explains tool calling.

5. 5

   Turn the dial down

   For facts, use a low temperature. It will not fix wrong knowledge, but it stops long-shot guesses.

6. 6

   Keep a human in the loop

   For health, money, law and anything you will publish, a person checks before it counts.