Design an agentic AI system for new tasks
PremiumLLM systemHard
- Agent loop
- Tool retrieval
- Context management
Quick answer
This LLM system interview question asks you to design an agentic AI system for new tasks. Aim for a design that uses language models to serve users well, then explain retrieval, prompting, serving, evaluation and trade-offs. Interviewers want to see how you control quality, cost and latency, and how you evaluate the results.
The problem
An agentic AI system is a platform that runs AI agents on new tasks nobody wrote a workflow for: a user states a goal and the agent plans, calls tools and reads the results until the goal is met. The hard part is that each task is a long, costly, non-deterministic job that must survive crashes, stay safe and stay within budget.
The platform serves many internal teams and external customers. Goals look like "find out why last night's billing job failed and open a ticket with the fix" or "compare these three vendor contracts and draft a summary". Tools include search, internal APIs, code execution, ticketing and email. This question has been reported at Anthropic in candidate interviews for senior and staff roles.
A task is a process, not a request. One task can make 20 to 100 model calls over several minutes, its context grows with every tool result, it may wait a day for a human to approve an action, and a bug in the loop can burn money or take an irreversible action. You are designing the runtime that runs thousands of these tasks at once reliably, safely and within budget, not a single clever prompt.
What must an agentic AI system for new tasks do?
- Plan, act, observe: The agent repeats model call, tool call and observation, revising its plan, until it returns a final result or hits a limit.
- Use new tools: A tool registered today with a schema and description is usable by new tasks within minutes, with no code deploy.
How well must it do it?
- Durability: No committed step is lost; after a worker crash a task resumes from its last committed step within 60 s, and no side-effecting tool call is applied twice.
- Cost: Average model spend of at most $0.50 per task at the assumed prices; every task has a hard ceiling (default $5, 100 steps, 30 minutes of active run time).
What scale should you design for?
Use these numbers in your estimates.
The scale numbers unlock with this questionUnlock
Key concepts
- Agent loop
- Tool retrieval
- Context management
- Durable execution
- Sandboxing
- Prompt caching
How you’ll practise
- Problem framing
- Data and retrieval
- Prompting and orchestration
- Evaluation and safety
- Serving, cost and latency
- Wrap-up
Lala, personalized to you, gives a hint when you ask and reviews your design when you are ready.
What do interviewers look for?
In any LLM system design round.
- Defines the user task and what a good answer is
- Explains how relevant context reaches the model
- Plans evaluation before and after launch
- Controls cost and latency with caching and routing
- Covers safety, privacy and abuse
- Handles failures and fallbacks gracefully
Questions people ask about this interview question
- Which design type is this question?
- It is an LLM system question. You design an agentic AI system for new tasks and explain your choices the way you would in an interview.
- Is this question asked at entry level?
- It is marked Hard. It suits engineers preparing for senior interviews, where you are expected to drive the discussion and go deep on trade-offs.
- What should I cover first?
- Start with the user task, what a good answer looks like and how it will be measured. Agree on scope with the interviewer before you design anything.
- Can I practise it for free?
- It is a premium question. Sign up and unlock it once for 10 credits; it then stays unlocked. When you finish, Lala, your personalized AI mentor, can review your design for 30 credits.
Related questions
Updated · Written by the SevyDevy team