Design a chatbot system
PremiumLLM systemHard
- Nlp
- AI
- Real time
Quick answer
This LLM system interview question asks you to design a chatbot system. Aim for a design that uses language models to serve users well, then explain retrieval, prompting, serving, evaluation and trade-offs. Interviewers want to see how you control quality, cost and latency, and how you evaluate the results.
The problem
Design a chatbot system that processes user messages, understands intent, and returns relevant responses through predefined rules, retrieval, or model-based generation.
What must a chatbot system do?
- Users can send messages to the chatbot in real time.
- The system should detect user intent from input text.
6 more requirementsUnlock to see all
How well must it do it?
- Chat responses should be low latency.
- The system should scale for many concurrent conversations.
4 more requirementsUnlock to see all
What scale should you design for?
Use these numbers in your estimates.
The scale numbers unlock with this questionUnlock
Key concepts
- Nlp
- AI
- Real time
- Scalability
- Intent recognition
- Response generation
How you’ll practise
6 short steps · save and come back any time
- Problem framing
- Data and retrieval
- Prompting and orchestration
- Evaluation and safety
- Serving, cost and latency
- Wrap-up
Lala, personalized to you, gives a hint when you ask and reviews your design when you are ready.
What do interviewers look for?
In any LLM system design round.
- Defines the user task and what a good answer is
- Explains how relevant context reaches the model
- Plans evaluation before and after launch
- Controls cost and latency with caching and routing
- Covers safety, privacy and abuse
- Handles failures and fallbacks gracefully
Questions people ask about this interview question
- Which design type is this question?
- It is an LLM system question. You design a chatbot system and explain your choices the way you would in an interview.
- Is this question asked at entry level?
- It is marked Hard. It suits engineers preparing for senior interviews, where you are expected to drive the discussion and go deep on trade-offs.
- What should I cover first?
- Start with the user task, what a good answer looks like and how it will be measured. Agree on scope with the interviewer before you design anything.
- Can I practise it for free?
- It is a premium question. Sign up and unlock it once for 10 credits; it then stays unlocked. When you finish, Lala, your personalized AI mentor, can review your design for 30 credits.
Related questions
Updated · Written by the SevyDevy team