Design an AI customer support agent
PremiumLLM systemEasy
- RAG
- Tool calling
- Human handoff
Quick answer
This LLM system interview question asks you to design an AI customer support agent. Aim for a design that uses language models to serve users well, then explain retrieval, prompting, serving, evaluation and trade-offs. Interviewers want to see how you control quality, cost and latency, and how you evaluate the results.
The problem
Design an AI customer support agent for an online electronics store. It chats with customers, answers from help-center articles, checks orders through tools and hands off to a human when needed. The hard parts are never inventing a policy, never touching another customer's order, and knowing when to hand off.
Customers are logged in when they open the chat, so the system knows who they are. Human agents already work in a separate console that receives handed-off chats.
What must an AI customer support agent do?
- Grounded answers: Answer policy questions (returns, warranty, shipping) only from the 3,000 help-center articles, with a link to the article used.
- Order help: Show order status and start a return for the logged-in customer's own orders when the policy allows it.
How well must it do it?
- Correctness: No answer states a policy that is not in the help articles; at least 90% correct resolutions on the evaluation set.
- Privacy: The agent sees only the logged-in customer's data; card numbers, phone numbers and addresses are masked in logs.
What scale should you design for?
Use these numbers in your estimates.
The scale numbers unlock with this questionUnlock
Key concepts
- RAG
- Tool calling
- Human handoff
- Prompt injection
- Guardrails
- PII masking
How you’ll practise
- Problem framing
- Data and retrieval
- Prompting and orchestration
- Evaluation and safety
- Serving, cost and latency
- Wrap-up
Lala, personalized to you, gives a hint when you ask and reviews your design when you are ready.
What do interviewers look for?
In any LLM system design round.
- Defines the user task and what a good answer is
- Explains how relevant context reaches the model
- Plans evaluation before and after launch
- Controls cost and latency with caching and routing
- Covers safety, privacy and abuse
- Handles failures and fallbacks gracefully
Questions people ask about this interview question
- Which design type is this question?
- It is an LLM system question. You design an AI customer support agent and explain your choices the way you would in an interview.
- Is this question asked at entry level?
- It is marked Easy, so it suits engineers who are new to LLM system interviews. It is also a good warm-up before harder questions.
- What should I cover first?
- Start with the user task, what a good answer looks like and how it will be measured. Agree on scope with the interviewer before you design anything.
- Can I practise it for free?
- It is a premium question. Sign up and unlock it once for 10 credits; it then stays unlocked. When you finish, Lala, your personalized AI mentor, can review your design for 30 credits.
Related questions
Updated · Written by the SevyDevy team