Why doesn’t ChatGPT know my company’s data?

Because your company’s documents were never in its training data. Chat apps like ChatGPT or Gemini learn from public text up to a cut-off date. To answer about your data, the app must be given that data, for example through an uploaded file or a connected document search.

What is a knowledge cut-off in AI?

It is the date a model’s training data ends. The model has no memory of events after it. Some apps add web search on top, which fetches fresh pages for each question, but the model itself does not learn from them.

Can I train AI on my own data instead?

You can, through fine-tuning, but it is usually not the best way to add facts. It costs time and money to redo whenever the facts change, and the model can still mix them up. Most teams start with RAG, which looks facts up at question time. The last lesson of this module compares the options.

Is it safe to paste private data into AI tools?

It depends on the tool and on your organisation’s rules. Some services may store or review what you send. Read the tool’s data policy, follow your workplace rules, and keep passwords and personal details out.

Module 4 · Making AI know your stuff · Lesson 18 of 32

Why AI doesn’t know your data

8 minFree GenAI course · BeginnerLesson 4.1By Saket Bhatnagar

Researched and drafted with AI, reviewed and edited by Saket Bhatnagar.

In short

A model only knows what was in its training data, and that data stops at a cut-off date.
Your company policies, your notes and today’s news were never in it, so it cannot know them by itself.
The fix is to hand it the right facts at question time, inside the prompt. The rest of this module shows how.

Soon

Video lesson · coming soon

Why AI doesn’t know your data, in 3 minutes

Lala, your AI mentor, will walk you through this lesson in a short animated video. Until then, everything is here to read and try.

01 The confident wrong answer
02 A topper who never saw your notice board
03 Where a model’s knowledge stops
04 Private, fresh, or very local
05 Can it know this by itself?
06 Next: the open-book exam

Meera joins a new job in Hyderabad. On day two, she asks a chat app, “How many casual leaves do I get at my company?” It replies in a second: “Most companies give 12 casual leaves a year.” Confident. Polite. And it has no idea. Her company gives eight.

The app did not lie on purpose. It never saw Meera’s HR policy, because nobody ever showed it. This lesson is about that gap. It is the most common reason AI tools disappoint people at work.

What does a model actually know?

Think back to how models are trained. Before a chat app ever talks to you, its model reads a huge pile of text: public web pages, books, articles and code. From that reading, it learns the patterns of language, and a lot of facts along the way.

Then the reading stops. The model is frozen and shipped. Everything it “knows” is whatever soaked into its settings during that reading. It does not keep reading the news after that.

The cut-off is usually several months, sometimes more than a year, before the day you use the model. So a model you chat with today may never have heard of last month’s budget, this week’s match or a phone launched yesterday.

Think of it like this

The topper who never saw your notice board

Kabir topped his state board. He has read every textbook, every guide and every past paper. Now he visits your college in Indore as a guest. You ask him, “When is our fest this year? Is the canteen open on Sunday?” Kabir is brilliant, but he has never stood in front of your college notice board. If he is honest, he says “I don’t know.” If he wants to please you, he guesses something that sounds right.

In real lifeIn AI

Kabir’s textbooks and guides: In AI: The model’s training data
The day he finished studying: In AI: The knowledge cut-off
Your college notice board: In AI: Your private documents and today’s updates
A confident guess about the fest date: In AI: A hallucination
Pinning the notice right in front of him: In AI: Putting your facts in the prompt

Three kinds of facts it can never know by itself

Most questions a model fails at fall into three buckets. Once you can spot them, you will know before you ask whether an answer deserves your trust.

What a model can and cannot know without help
Kind of fact	Example	Why the model misses it
Private	Your company’s leave policy, your class notes, a client’s order history	It was never public, so it was never in the training data.
Fresh	Today’s cricket score, this morning’s petrol price, a rule announced last week	It happened after the knowledge cut-off.
Rare or very local	Your society gym’s timings, a small shop’s return rule	It appeared online once or twice at most, too rarely to be learned well.
General and stable	What photosynthesis is, how to write a leave letter	Usually fine: it appeared thousands of times in the training data.

Look at the last row. Models are very good at general, stable knowledge. The trouble starts only when the answer lives somewhere the model never looked.

What happens when you ask anyway?

Remember the hallucinations lesson: a model is trained to produce text that sounds right. It has no little light that turns red when a fact is missing. So when you ask about your private data, it fills the gap with the most likely-sounding answer.

That is how Meera got “12 casual leaves”. Twelve is a common number in many policies, so it sounds likely. It is not her company’s answer.

Where a model’s knowledge comes from

Drag to explore

The model learns only from the training pile, once. Your files and today’s news never join that pile, so a question about them gets a guess.

Some chat apps now search the web, or read a file you upload, before they answer. That helps a lot. But notice what is happening: the app fetches fresh text and hands it to the model. The model itself still knows nothing new. Hold on to that idea. It is the whole trick of the next lesson.

So how do you give AI your data?

If the model cannot know your facts, you bring the facts to it. The plainest way is to paste them into the prompt. Copy the leave policy, paste it, then ask your question. The model reads it like any other text, and answers from it.

Pasting works, and it works today. But you met its limits in the context window lesson. The window holds only so many tokens. Every pasted page costs money and time, on every message. And nobody wants to paste a three-hundred-page handbook before each question.

So real apps do something smarter. They keep your documents in a searchable store, find only the few paragraphs that answer this question, and paste those in for you. That is called RAG, and it is the next lesson.

Try it yourselfCan the model know this by itself?For each question, decide: is the model’s training enough, or does it need you to give it the facts?

Check yourself

Riya asks a chat app about a rule her college announced last week. It answers confidently. What is the most likely problem?

Pick an answer to see why it is right or wrong.

Check yourself

Which of these lets a model answer correctly about your company’s leave policy?

Pick an answer to see why it is right or wrong.

Remember these three

A model knows only its training data, which stops at a knowledge cut-off.
Private, fresh and very local facts were never in it, so it guesses fluently.
To get right answers about your stuff, put the facts in front of it at question time.

Questions people ask

Why doesn’t ChatGPT know my company’s data?: Because your company’s documents were never in its training data. Chat apps like ChatGPT or Gemini learn from public text up to a cut-off date. To answer about your data, the app must be given that data, for example through an uploaded file or a connected document search.
What is a knowledge cut-off in AI?: It is the date a model’s training data ends. The model has no memory of events after it. Some apps add web search on top, which fetches fresh pages for each question, but the model itself does not learn from them.
Can I train AI on my own data instead?: You can, through fine-tuning, but it is usually not the best way to add facts. It costs time and money to redo whenever the facts change, and the model can still mix them up. Most teams start with RAG, which looks facts up at question time. The last lesson of this module compares the options.
Is it safe to paste private data into AI tools?: It depends on the tool and on your organisation’s rules. Some services may store or review what you send. Read the tool’s data policy, follow your workplace rules, and keep passwords and personal details out.