How is the cost of an LLM API calculated?

You are charged per token, usually quoted per million tokens, with separate prices for input and output. Multiply the tokens you send and receive by their prices, then by how many calls you make. Output tokens usually cost more.

How much does it cost to run an AI chatbot?

It depends on users, message length, chat history and the model. With example prices, a thousand daily users can cost from a few thousand rupees a month on a small model to tens of thousands on a large one. Real prices vary by model and change often.

How can I reduce LLM costs?

Send fewer tokens and generate fewer tokens. Trim the system prompt and history, retrieve fewer but better chunks, cap answer length, cache repeated questions and route simple tasks to smaller models.

Why do Hindi prompts sometimes use more tokens?

Tokenizers learn their pieces mostly from training text, and many have seen more English than Indian languages. So a Hindi sentence is often split into more, smaller tokens. Newer tokenizers are narrowing this gap.

Module 7 · Building real AI products · Lesson 28 of 32

Cost and speed: the token meter

10 minFree GenAI course · BeginnerLesson 7.2By Saket Bhatnagar

Researched and drafted with AI, reviewed and edited by Saket Bhatnagar.

In short

You pay for every token going in and every token coming out, like a meter that never stops ticking.
Input grows quietly: system prompt, chat history and documents are sent again with every message.
Shorter prompts, capped answers, caching and smaller models for simple jobs cut both the bill and the wait.

Soon

Video lesson · coming soon

How much does AI cost? The token meter, explained

Lala, your AI mentor, will walk you through this lesson in a short animated video. Until then, everything is here to read and try.

01 Same students, a much bigger bill
02 An auto with two meters
03 The question is the smallest bag
04 Run the token meter yourself
05 Tokens cost time too
06 Four ways to shrink the bill
07 Play with the meter, then test quality

Arjun runs a coaching centre in Indore and adds an AI doubt helper for his students. In the first week, the bill is a few hundred rupees. By the end of the month, it is many times that, with the same number of students. Nobody changed the model. So where did the money go?

The answer is hiding in the tokens. Once you can see how they add up, you can predict the bill before it arrives, and shrink it.

What exactly are you paying for?

You met tokens early in this course: the small pieces of text a model reads and writes. Model services charge by the token, usually quoted as a price per million tokens. There are two meters running at once.

Output tokens usually cost more than input tokens, often several times more, because the model produces them one at a time. Reading a long prompt is fast work for the computer; writing each new token is slower work.

Think of it like this

An auto with two meters

Imagine an auto-rickshaw with two meters. One ticks for every bag you load before the ride. The other ticks, faster, for every kilometre driven. Load less luggage and you pay less before you even start. Ask to be dropped at the corner instead of the far gate, and the second meter stops sooner. And for a short trip, a shared auto does the job at a fraction of a premium cab’s fare.

In real lifeIn AI

Bags loaded before the ride: In AI: Input tokens: system prompt, history, documents and the question
Kilometres driven, ticking faster: In AI: Output tokens, usually priced higher
Shared auto or premium cab: In AI: A small, cheap model or a large, costly one
“Drop me at the corner”: In AI: Capping how long the answer can be
Leaving the heavy suitcase at home: In AI: Trimming the prompt to what the model needs

A useful rule of thumb for English: one token is roughly three-quarters of a word, so a hundred words is about 130 tokens. Hindi, Tamil and other Indian languages often need more tokens for the same sentence, depending on the tokenizer. That means the same chat can cost more in some languages.

Where do all the input tokens come from?

Here is the surprise behind Arjun’s bill. The student’s question is often the smallest part of the prompt. Remember from the last lesson that the server builds each prompt from several pieces. Every one of them is counted, every single time.

What one model call is billed for

Drag to explore

Four kinds of input are sent with every message, and they are all counted. The answer is counted separately, usually at a higher price.

Chat history is the quiet one. The model has no memory, so the app resends the conversation each time. Message ten carries the nine before it. Here is what that looks like with a 400-token system prompt and about 150 tokens per back-and-forth.

Input tokens per message in one long chat (example numbers)
Message in the chat	Input tokens sent	Compared with message 1
Message 1	About 450	Same
Message 5	About 1,050	More than double
Message 10	About 1,800	Four times
Message 20	About 3,300	More than seven times

That is exactly what happened to Arjun. As students got comfortable, their chats grew longer. Same students, same model, but each message carried far more luggage.

How do you estimate a monthly bill?

You do not need a spreadsheet to start. Multiply four things: users per day, messages per user, tokens per message, and days in the month. Then multiply by the price. Here is a worked example with made-up but realistic sizes.

text

Optional: a back-of-the-envelope estimate

1,000 students x 5 messages a day   = 5,000 messages a day
each message: 800 tokens in, 200 tokens out

per month (30 days):
  input  = 5,000 x 800 x 30 = 120 million tokens
  output = 5,000 x 200 x 30 =  30 million tokens

EXAMPLE prices only (real ones vary by model and change often):
  small model: ₹20 per million in, ₹80 per million out
    120 x ₹20 + 30 x ₹80     = ₹2,400 + ₹2,400  = about ₹4,800 a month
  large model: ₹250 per million in, ₹1,000 per million out
    120 x ₹250 + 30 x ₹1,000 = ₹30,000 + ₹30,000 = about ₹60,000 a month

Same students, same questions, and one choice changes the bill more than ten times. Those prices are examples for learning, not quotes. Real prices differ a lot between models and services, and they change often, usually downwards. Always check the current price page of whatever you use.

The starting settings match the worked example: 1,000 people, 5 messages a day, 120 million tokens in and 30 million out. At an example ₹100 per million tokens, that is about ₹15,000 a month. Double the people and the bill doubles. Double the words sent in and it almost doubles, because input is most of the tokens.

Now switch on caching. The calculator treats half of each prompt as a repeat, like a fixed system prompt, and charges those tokens about a tenth. The bill drops to about ₹9,600. The calculator uses one example price for all tokens to keep things simple; real services usually charge more for output.

Why do some answers feel slow?

Tokens cost time as well as money. Before the first word appears, the model has to read the whole prompt. Then it writes the answer one token at a time. A long prompt delays the start; a long answer delays the finish.

Speeds vary a lot by model and service, but think in tens of tokens per second, sometimes a few hundred. A 500-token answer at 50 tokens a second takes ten seconds to finish. That is why streaming matters: the student starts reading in about a second, even if the full answer takes longer.

How do you cut the bill and the wait?

The good news: the same tricks usually make the app cheaper and faster at once. Fewer tokens means less to pay for and less to wait for.

Trim the prompt. Cut extra words from the system prompt, send only the last few messages or a short summary of older ones, and pass three good document chunks instead of ten.
Cap the answer. Set a maximum answer length and ask for short replies, like “answer in under 80 words”. This also cuts the most expensive tokens.
Pick the right size of model. Use a small, fast model for simple jobs like sorting questions or answering FAQs, and send only hard questions to a big model. Choosing per request is called routing.
Cache repeats. If 300 students ask “What time does the centre open?”, store the answer once and reuse it. Many services also offer prompt caching, which charges much less for a repeated prompt beginning.
Set limits. A cap on messages per user per day protects you from one runaway user or bug.

Check yourself

Arjun has the same number of students, but his monthly bill tripled. The app now sends the full chat history with every message. What is the most likely cause?

Pick an answer to see why it is right or wrong.

Check yourself

Hundreds of students ask “What time does the centre open?” every day. What is the cheapest good fix?

Pick an answer to see why it is right or wrong.

Check yourself

Students wait about 15 seconds for each answer. Which change helps the most?

Pick an answer to see why it is right or wrong.

Remember these three

Cost = tokens in x input price + tokens out x output price, for every message, every day.
The hidden cost is the prompt: system prompt, history and documents resent each time.
Trim prompts, cap answers, cache repeats and use small models for simple jobs, then check quality did not drop.

Questions people ask

How is the cost of an LLM API calculated?: You are charged per token, usually quoted per million tokens, with separate prices for input and output. Multiply the tokens you send and receive by their prices, then by how many calls you make. Output tokens usually cost more.
How much does it cost to run an AI chatbot?: It depends on users, message length, chat history and the model. With example prices, a thousand daily users can cost from a few thousand rupees a month on a small model to tens of thousands on a large one. Real prices vary by model and change often.
How can I reduce LLM costs?: Send fewer tokens and generate fewer tokens. Trim the system prompt and history, retrieve fewer but better chunks, cap answer length, cache repeated questions and route simple tasks to smaller models.
Why do Hindi prompts sometimes use more tokens?: Tokenizers learn their pieces mostly from training text, and many have seen more English than Indian languages. So a Hindi sentence is often split into more, smaller tokens. Newer tokenizers are narrowing this gap.