Attention and transformers, without maths

Late one night, Priya’s nani posts on the family WhatsApp group: “Dal is on the stove. Put it in the fridge once it cools.” Nobody replies, “Wait, put the stove in the fridge?”

Everyone knows “it” means the dal. Nobody looked up a grammar rule. Their eyes went back to the right word in a split second. That quick glance back, picking out the words that matter, is what AI researchers call attention.

Many words only make sense with their neighbours. “Sharp” means one thing in “a sharp knife”, another in “a sharp student”, and another in “8 o’clock sharp”. “It” means nothing at all until you know what it points to.

In the last lesson, every token got an embedding: a starting spot on the map of meaning. But that starting spot is the same every time. “Bank” gets one vector whether you are sitting on a river bank or applying for a bank loan. The model needs a way for each word to update its meaning using the words around it.

Here is the whole move, without maths. Each word holds up its query. It compares that query with every word’s key and gets a score for each. The scores are turned into shares that add up to 100%. Then the word takes a blend of everyone’s values, mostly from the high scorers. Its meaning is now updated by its context.

This classic test sentence trips up anything that reads words one by one without context: “The trophy did not fit in the bag because it was too big.” You know “it” is the trophy. Change one word, “…because it was too small”, and “it” is suddenly the bag.

• With the “too big” ending, tap “it”. The strongest share, about 58%, goes to “trophy”.
• Switch to “too small” and tap “it” again. Now most of its attention moves to “bag”.
• With each ending, tap the last word, “big” or “small”. It looks back at the object it describes.
• Tap a small word like “in” or “did”. It mostly looks at itself and its neighbours.

One round of attention is one glance. Real models take many glances side by side, each with its own query, key and value. Researchers have found some that seem to track which word a pronoun refers to, some that link a verb to its subject, and some that watch nearby words. Many others do jobs nobody has neatly named.

After the words have gathered information from each other, each word goes through a small step on its own, called a feed-forward network. You can think of attention as gathering notes from classmates, and the feed-forward step as sitting quietly and making sense of them. Much of what the model knows about the world seems to be stored in these steps.

Notice the position marker. Attention on its own has no sense of order, so “dog bites man” and “man bites dog” would look the same. Adding a position signal to each embedding lets the model tell them apart.

Then the same kind of block is stacked again and again: dozens of layers in large models, sometimes more than a hundred. Roughly speaking, early layers pick up local patterns like word pieces and grammar, and later layers build more abstract meaning. By the top, “it” in our sentence carries a strong dose of “trophy”, and “bank” in “bank loan” has moved far from the river.

Before 2017, popular language models were recurrent neural networks. They read text one word at a time, carrying a running summary forward, a bit like passing a note down a long row of students. By the end of a long paragraph, the early words had faded. And you could not speed them up much, because each step had to wait for the one before.

In 2017, a team of researchers at Google published a paper called “Attention Is All You Need”. They dropped the word-by-word reading and built the model from attention alone. Every word could look directly at every other word, and during training all the words could be processed at once.

That “all at once” design fit GPUs perfectly. Training on far larger piles of text became practical, and the models got far better. The “T” in GPT stands for transformer: generative pre-trained transformer. Today, transformers or close cousins power most chat AI, translation tools and many image and speech models too.

That last row matters. Because every token looks at every other token, doubling the length of the text roughly quadruples the attention work. It is one reason every model has a limit on how much text it can take in at once. You’ll meet that limit, the context window, later in this module. Engineers have found many clever tricks to soften that cost, but it never fully goes away.