What is a context window in AI?

It is the maximum amount of text, counted in tokens, that a language model can look at in one go. It includes the instructions, the conversation so far, any documents and the reply the model writes.

Why does ChatGPT forget what I said earlier?

In a long chat, the earliest messages may no longer fit in the context window, so the app drops or summarises them. The model cannot use what it is not sent, even though you can still see it on screen.

How big is a context window?

It depends on the model. Older and smaller models may hold only a few thousand tokens, and many popular models in 2026 hold roughly 128 thousand to a million tokens. In English, a token is about three quarters of a word.

Is a bigger context window always better?

Not always. Bigger inputs cost more, answer more slowly, and models can miss details in the middle of very long text. Sending only the relevant parts often gives better answers.

Module 2 · How a language model thinks · Lesson 11 of 32

Context window: how much AI can keep in mind

9 minFree GenAI course · BeginnerLesson 2.7By Saket Bhatnagar

Researched and drafted with AI, reviewed and edited by Saket Bhatnagar.

In short

The context window is the most tokens a model can look at in one go: instructions, chat history, documents and its own reply.
The model has no memory of its own. The app resends the conversation every turn, and whatever does not fit is dropped.
Long chats forget early details, cost more and get slower; summaries, search and fresh chats keep things sharp.

Soon

Video lesson · coming soon

What is a context window? Why AI forgets

Lala, your AI mentor, will walk you through this lesson in a short animated video. Until then, everything is here to read and try.

01 Why did the AI forget I’m vegetarian?
02 A blackboard with fixed space
03 Everything shares one window
04 The model never remembered
05 Watch the first message slide out
06 Bigger windows have a price
07 Try it, then learn why AI makes things up

Meera spends an evening planning a Goa trip with a chat app. In her very first message she says, “I am vegetarian.” Two hours and a hundred messages later, the app happily suggests a famous prawn curry shack. She scrolls up. The message is still there on her screen. So why did the AI forget it?

Because what you see on your screen and what the model sees are two different things. The model only sees what fits in its context window.

What is a context window?

Every time a model answers, it reads a block of text and predicts what comes next, token by token. That block has a size limit, counted in tokens, not words or messages. The limit is fixed by how the model was built and trained.

Think of it like this

The tuition teacher’s blackboard

Asha ma’am teaches maths at a tuition centre in Kanpur. Her blackboard is a fixed size. As she solves a long problem, she keeps writing at the bottom, and when the board is full she wipes the oldest lines at the top to make room. A student who looks up now can only use what is on the board. The first line, “Let x be the number of mangoes”, was wiped ten minutes ago, so the last step makes no sense. Clever students copy a short summary into their notebook, or open the textbook to the right page when needed.

In real lifeIn AI

The size of the blackboard: In AI: The context window, measured in tokens
Lines of chalk on the board: In AI: Instructions, messages and documents sent to the model
Wiping the oldest lines: In AI: The app dropping old messages that no longer fit
A short summary in the notebook: In AI: Summarising older chat into a few lines
Opening the textbook to the right page: In AI: Searching documents and adding only the relevant part

Sizes vary a lot. Older and smaller models may hold only a few thousand tokens. Many popular models in 2026 hold roughly 128 thousand to a million tokens, and a few go beyond that. In English, one token is about three quarters of a word, so 128 thousand tokens is roughly a whole novel.

What fills the window?

People think the window holds “my chat”. It holds a lot more. Before your words even arrive, the app has usually added hidden instructions: who the assistant is, rules to follow, the date. If you attached a PDF or the app searched the web, that text goes in too.

Everything that shares one context window

Drag to explore

All of these come out of the same token budget, and the reply needs room as well.

The reply counts too. If the window is nearly full of history, there is little room left to answer, and the reply may be cut short. Apps often set a separate cap on reply length, so a long essay may stop mid-sentence even when the window has space.

Why does AI forget? Because it never remembered

Here is the part that surprises most people. The model does not remember your last message at all. Each time you press send, the app packs the hidden instructions, the whole chat so far and your new message into one block, and sends it fresh. The model reads it from scratch, answers, and forgets everything.

Think of a chai-wala on a busy station platform who forgets every face the moment the train leaves. If you want your usual “kam cheeni, adrak wali”, you have to say it again each time. Apps do that repeating for you. As the chat grows, the block grows, until it no longer fits. Then the app must drop something, usually the oldest messages.

That is what happened to Meera. Her “I am vegetarian” was the oldest message, so it was the first to slide out. On her screen it is still there. For the model, it did not exist in this turn.

Ask early, and the reply is “Riya’s birthday is on 12 March.” Ask after a few more messages, and the first message turns grey, above a line that says the model can only see below it. Now the reply is “I’m not sure. You haven’t told me Riya’s birthday.” Nothing was wrong with the model. The fact was outside its window, so for the model it was not there at all.

Bigger is better, right? Not always

A huge window sounds like the fix. It helps, but it brings three costs.

Money: you pay for every token in, and long chats resend the whole history every turn. The hundredth message carries the ninety-nine before it.
Time: more tokens to read means a slower first word.
Focus: models tend to use the start and end of a long input well and miss details buried in the middle.

So a 300-page PDF that fits in the window is not the same as a 300-page PDF the model reads carefully. Newer models handle long inputs better than older ones, but sending only what matters still gives sharper answers.

Ways apps and users deal with a full window
Approach	How it works	Trade-off
Drop the oldest	Keep the latest messages, cut the earliest	Simple, but early facts vanish
Summarise	Turn old messages into a short recap and keep that	Saves space; small details can be lost
Search and insert	Store documents outside, add only the relevant parts	Sharp and cheap; needs a good search (module 4)
Start a fresh chat	Begin again with a short brief of what matters	Clean and fast; you write the brief

Check yourself

Kabir’s chat is very long. The AI ignores a rule he set in the first message. What is the most likely reason?

Pick an answer to see why it is right or wrong.

Check yourself

Which of these does NOT use up space in the context window?

Pick an answer to see why it is right or wrong.

Check yourself

Why does message number 100 in a chat usually cost more than message number 1?

Pick an answer to see why it is right or wrong.

Remember these three

The context window is the model’s whole view: instructions, history, documents and its reply, all counted in tokens.
The model is stateless; the app resends the chat each turn and drops what no longer fits, which is why AI “forgets”.
Bigger windows cost more, run slower and can lose details in the middle; summaries, search and fresh chats help.

Questions people ask

What is a context window in AI?: It is the maximum amount of text, counted in tokens, that a language model can look at in one go. It includes the instructions, the conversation so far, any documents and the reply the model writes.
Why does ChatGPT forget what I said earlier?: In a long chat, the earliest messages may no longer fit in the context window, so the app drops or summarises them. The model cannot use what it is not sent, even though you can still see it on screen.
How big is a context window?: It depends on the model. Older and smaller models may hold only a few thousand tokens, and many popular models in 2026 hold roughly 128 thousand to a million tokens. In English, a token is about three quarters of a word.
Is a bigger context window always better?: Not always. Bigger inputs cost more, answer more slowly, and models can miss details in the middle of very long text. Sending only the relevant parts often gives better answers.