What is the architecture of an LLM application?

Most LLM apps have an interface, a backend server, a store for chat history and documents, the model, guardrails, and logging. The server checks each request, builds the prompt with retrieved context, calls the model and checks the reply.

Do I need to train my own model to build an AI app?

Usually not. Most apps call an existing model through an API and get their special knowledge from a good system prompt and RAG over their own documents. Training or fine-tuning comes much later, if ever.

How does a chatbot remember earlier messages?

It does not remember by itself. The app saves the conversation and sends recent messages back with each new question. Very long chats are trimmed or summarised so they fit the context window.

What are guardrails in AI apps?

Guardrails are checks around the model. Input checks block abuse, tricks and private data; output checks make sure answers are safe, on topic and in the right format before users see them.

Module 7 · Building real AI products · Lesson 27 of 32

The anatomy of an AI app

12 minFree GenAI course · BeginnerLesson 7.1By Saket Bhatnagar

Researched and drafted with AI, reviewed and edited by Saket Bhatnagar.

In short

A real AI app is many parts working around one model call, not the model alone.
A server checks each message, builds the prompt from history and documents, calls the model, and checks the answer.
Guardrails, logs and tests turn a nice demo into something people can trust every day.

Soon

Video lesson · coming soon

How AI apps are really built, in 3 minutes

Lala, your AI mentor, will walk you through this lesson in a short animated video. Until then, everything is here to read and try.

01 The demo worked. Then real users came.
02 The model is the cook, not the dhaba
03 Six parts around one model call
04 Follow one message through the app
05 Keep the secret key on the server
06 Start small, add parts as you grow
07 Try the stepper, then count the cost

Priya builds a helper for her college library in one evening. She types “Can I borrow three books?” and the model answers perfectly. She shares the link in the class WhatsApp group and goes to sleep happy.

By morning, things have gone sideways. It told one student the wrong late fee. It forgot a name someone gave it two messages earlier. Someone made it write rude poems. And her free credits are gone. Nothing was wrong with the model. Everything around it was missing.

Why is an AI app more than a model?

The model is the clever part, but it is only one part. It reads a prompt and writes text. That is all it does. It does not know who is asking, what was said yesterday, what your library rules are, or how much money you have left.

Everything else is ordinary software that you build around it. Think of a busy dhaba. The cook is brilliant, but customers never walk into the kitchen and shout at the cook. A whole system sits in between.

Think of it like this

The dhaba behind your dal

At Kabir’s favourite highway dhaba, a waiter takes your order and writes it on a slip. The manager checks the slip, adds “less spicy, regular customer”, and passes it in. The cook makes the dish using the dhaba’s own recipes and pantry. Before the plate leaves, someone checks it looks right. The manager writes every order in a register, so tomorrow they know what sold and what went wrong.

In real lifeIn AI

Waiter and menu card: In AI: The interface: the chat box, app or website
Manager with the order slip: In AI: The server that checks requests and builds the prompt
Recipes and pantry: In AI: The system prompt, chat history and your documents
The cook: In AI: The model, which writes the answer
Plate check and order register: In AI: Guardrails, logs and tests

The parts of a typical AI app

Drag to explore

The user only ever talks to the interface. The server does the real work: it checks the message, gathers history and documents, calls the model, checks the answer and writes a log.

What happens to one chat message?

Let us follow one message through Priya’s fixed-up app. A student asks: “Is Wings of Fire available, and can I keep it for three weeks?” It looks like one question. Behind the screen, about seven things happen in a second or two.

Try it yourselfThe journey of one chat messageStep through what the app does between your tap on “send” and the answer on your screen.

Which parts make it safe and reliable?

A demo only needs to work once, for you. A product has to work for thousands of strangers, on bad days too. Most of the extra work goes into four jobs.

Guardrails. On the way in: block abuse, hide private details, spot tricks like “ignore your rules”. On the way out: check the answer is safe, on topic and in the right shape, such as valid JSON from the structured-output lesson.
Limits. A cap on messages per user per hour, a maximum answer length and a timeout. These stop one person, or one bug, from emptying your wallet.
Fallbacks. Model services sometimes slow down or fail. A good app retries once, can switch to a backup model, and shows a polite message instead of a blank screen.
Logs and feedback. Every call records how long it took, how many tokens it used and whether the user liked it. You cannot fix what you cannot see.

The same helper as a demo and as a product
Part	One-evening demo	Real product
Who uses it	You, a few times	Hundreds of strangers, all day
Prompt	Typed by hand	Built by the server from rules, history and documents
Memory	None, or the whole chat pasted in	Saved history, trimmed to fit the context window
Safety	Hope for the best	Input and output guardrails
Cost control	Free credits	Rate limits, answer length caps, caching
Knowing it works	It answered my three questions	A test set run after every change, plus live logs

Two of those rows get their own lessons next. Cost comes up straight after this one. Testing comes after that. For now, notice how little of the table is about the model itself.

Do you need to build all of this on day one?

No. Start small and add parts when real use asks for them. Most teams follow roughly the same order, and each step fixes a problem Priya met on her first morning.

Prompt and server
Move the model call onto a server, hide the key, and write a clear system prompt.
Memory
Save chat history and send the recent part back each time, so the bot remembers names.
Your knowledge
Add RAG over your documents so answers come from your rules, not from guesses.
Limits and guardrails
Cap usage, cap answer length, and check what goes in and comes out.
Logs and tests
Record every call and keep a set of test questions to re-check after each change.
Tools, only if needed
Let the model look things up or act, like the catalogue check, once the basics are solid.

If you want to see the server part in code, here is an optional sketch. It is written like a recipe, not a real program, so you can read it without knowing any language.

text

Optional: the server's job, in plain steps

on new message from user:
  if not logged_in(user) or over_limit(user): refuse politely
  if input_check(message) fails: refuse politely
  history = load_last_messages(user, 4)
  rules   = search_documents(message, top = 3)
  prompt  = SYSTEM_PROMPT + rules + history + message
  answer  = call_model(prompt, max_answer_tokens = 300)
  if output_check(answer) fails: answer = safe_fallback
  save(user, message, answer)
  log(tokens, seconds, cost)
  stream answer to user

Check yourself

Priya’s bot forgets a student’s name two messages later. Which part of the app should fix this?

Pick an answer to see why it is right or wrong.

Check yourself

Why should the browser talk to your server instead of calling the model service directly?

Pick an answer to see why it is right or wrong.

Remember these three

The model is the cook; the app is the whole dhaba around it.
One message passes through checks, memory, search, a built prompt, the model, more checks and a log.
Start with a server and a clear prompt, then add memory, RAG, limits, guardrails, logs and tests as real use demands.

Questions people ask

What is the architecture of an LLM application?: Most LLM apps have an interface, a backend server, a store for chat history and documents, the model, guardrails, and logging. The server checks each request, builds the prompt with retrieved context, calls the model and checks the reply.
Do I need to train my own model to build an AI app?: Usually not. Most apps call an existing model through an API and get their special knowledge from a good system prompt and RAG over their own documents. Training or fine-tuning comes much later, if ever.
How does a chatbot remember earlier messages?: It does not remember by itself. The app saves the conversation and sends recent messages back with each new question. Very long chats are trimmed or summarised so they fit the context window.
What are guardrails in AI apps?: Guardrails are checks around the model. Input checks block abuse, tricks and private data; output checks make sure answers are safe, on topic and in the right format before users see them.