Design an AI search engine like perplexity
PremiumLLM systemMedium
- RAG
- Citations
- Web search
Quick answer
This LLM system interview question asks you to design an AI search engine like perplexity. Aim for a design that uses language models to serve users well, then explain retrieval, prompting, serving, evaluation and trade-offs. Interviewers want to see how you control quality, cost and latency, and how you evaluate the results.
The problem
Design an AI search engine like Perplexity. A user asks a question, the system searches the web, reads the best pages and streams a short answer where every claim cites a source. The hard parts are grounding each sentence in a page that really supports it, a first token within 2 seconds, and treating web pages as untrusted input.
The answer must be fresh enough for news questions, honest when sources disagree or say nothing, and cheap enough to serve 10 million questions a day.
What must an AI search engine like perplexity do?
- Cited answers: Answer each question in 100 to 300 words with inline numbered citations that link to the exact source pages.
- Live sources: Use current web results, so a question about this morning's news cites this morning's pages.
How well must it do it?
- Grounding: At least 90% of cited sentences are supported by the cited page on the evaluation set, and no URL appears unless it came from retrieval.
- Latency: First token within 2 s at p50 and 3.5 s at p95; full answer within 8 s at p95.
What scale should you design for?
Use these numbers in your estimates.
The scale numbers unlock with this questionUnlock
Key concepts
- RAG
- Citations
- Web search
- Reranking
- Prompt injection
- Answer caching
How you’ll practise
- Problem framing
- Data and retrieval
- Prompting and orchestration
- Evaluation and safety
- Serving, cost and latency
- Wrap-up
Lala, personalized to you, gives a hint when you ask and reviews your design when you are ready.
What do interviewers look for?
In any LLM system design round.
- Defines the user task and what a good answer is
- Explains how relevant context reaches the model
- Plans evaluation before and after launch
- Controls cost and latency with caching and routing
- Covers safety, privacy and abuse
- Handles failures and fallbacks gracefully
Questions people ask about this interview question
- Which design type is this question?
- It is an LLM system question. You design an AI search engine like perplexity and explain your choices the way you would in an interview.
- Is this question asked at entry level?
- It is marked Medium. Entry-level engineers can practise it, but it expects you to reason about requirements and trade-offs, not only list components.
- What should I cover first?
- Start with the user task, what a good answer looks like and how it will be measured. Agree on scope with the interviewer before you design anything.
- Can I practise it for free?
- It is a premium question. Sign up and unlock it once for 10 credits; it then stays unlocked. When you finish, Lala, your personalized AI mentor, can review your design for 30 credits.
Related questions
Updated · Written by the SevyDevy team