Design the OpenAI playground
PremiumProduct architectureMedium
- Token streaming
- SSE
- Prompt versioning
Quick answer
This product architecture interview question asks you to design the OpenAI playground. Aim for a design that meets its requirements at the stated scale, then explain its main components, data flow and trade-offs. Interviewers want to see how you scope the problem, size it and defend each choice.
The problem
The OpenAI Playground is a web app where developers write a system message and a multi-turn conversation, pick a model and parameters, and watch the reply stream in. The hard parts are runs you can reproduce exactly, immutable prompt versions, little added latency on streams and an API key that never reaches the browser.
A developer signs in, picks a project, writes the prompt, presses Run and then iterates: edit an earlier turn and re-run from there, add a fake assistant turn to simulate a conversation, run the same prompt with two parameter sets side by side, save the prompt as a version the team can reuse, and export the exact request as Python, JavaScript or curl code. This question has been reported at OpenAI in candidate interviews for full-stack roles.
The model API already exists and is a black box to you. It is a stateless HTTPS endpoint that takes messages plus parameters, streams tokens back, answers 429 when a project is over its rate limit, and ends each stream with a usage report of input and output tokens. Every call needs a project API key. You own the product around it: the screen, the backend API, the data model, the streaming path, and the rules that keep the API key secret and usage numbers right.
What must the OpenAI playground do?
- Run and stream: Pressing Run streams the reply token by token into the page, and a Stop button ends generation within one second.
- Prompt versions: A user can save a prompt as an immutable numbered version, view history, diff two versions, restore one, and mark one version as production.
How well must it do it?
- Secret safety: The API key secret never reaches the browser, logs, analytics, error reports or exported code.
- Added latency: The playground adds at most 100 ms p99 before the first token, on top of the model API's own time to first token.
What scale should you design for?
Use these numbers in your estimates.
The scale numbers unlock with this questionUnlock
Key concepts
- Token streaming
- SSE
- Prompt versioning
- Secret management
- Usage metering
- Relational modeling
How you’ll practise
- Requirements
- Core entities
- API design
- Client and server flow
- Scale and edge cases
- Wrap-up
Lala, personalized to you, gives a hint when you ask and reviews your design when you are ready.
What do interviewers look for?
In any product architecture design round.
- Clarifies the requirements before drawing anything
- Uses a few numbers that actually change the design
- Draws a clear path from the client to storage
- Picks storage and caching with reasons, not by habit
- Explains what breaks first under load and how to fix it
- States trade-offs openly instead of claiming a perfect design
Questions people ask about this interview question
- Which design type is this question?
- It is a product architecture question. You design the OpenAI playground and explain your choices the way you would in an interview.
- Is this question asked at entry level?
- It is marked Medium. Entry-level engineers can practise it, but it expects you to reason about requirements and trade-offs, not only list components.
- What should I cover first?
- Start with the requirements: what the system must do, how well it must do it and which numbers change the design. Agree on scope with the interviewer before you design anything.
- Can I practise it for free?
- It is a premium question. Sign up and unlock it once for 10 credits; it then stays unlocked. When you finish, Lala, your personalized AI mentor, can review your design for 30 credits.
Related questions
Updated · Written by the SevyDevy team