Design GPU telemetry analytics with NL-to-SQL
PremiumData pipelineHard
- GPU telemetry
- Time series rollups
- NL to SQL
Quick answer
This data pipeline interview question asks you to design GPU telemetry analytics with NL-to-SQL. Aim for a design that moves data from sources to consumers reliably and on time, then explain ingestion, processing, storage and trade-offs. Interviewers want to see how you handle volume, late or bad data and failures along the way.
The problem
Design GPU telemetry analytics for a fleet of 200,000 GPUs, with dashboards over 3 months, per-second drilldowns and an NL-to-SQL assistant. It resembles a fleet monitoring and analytics platform. The hard parts are storing two resolutions cheaply, answering long-range queries fast, and keeping generated SQL inside the permissions and cost limits of each user.
An agent on every GPU host sends 30 metrics per GPU (utilization, memory used, temperature, power, clocks, error counts, interconnect traffic) once per second. Engineers use dashboards for trends over weeks and drill into single jobs second by second when training slows down. They can also ask questions in plain English, such as "which of my jobs had the most idle GPU hours last week", and the assistant writes and runs SQL for them.
What must the system do?
- Two resolutions: Keep every metric at 1-second resolution for 7 days and at 30-second resolution for 90 days.
- Long-range dashboards: Show trends per GPU, job, project and cluster over any window up to 90 days.
How well must it do it?
- Permission scope: No dashboard, drilldown or assistant query ever returns rows for GPUs outside their own projects.
- Query speed: 90-day dashboards load in under 2 seconds and 10-minute drilldowns in under 1 second, at p95.
What scale should you design for?
Use these numbers in your estimates.
The scale numbers unlock with this questionUnlock
Key concepts
- GPU telemetry
- Time series rollups
- NL to SQL
- Row level security
- Query cost guard
- Kafka
How you’ll practise
- Requirements
- Data volume
- Ingestion
- Storage and processing
- Serving and freshness
- Wrap-up
Lala, personalized to you, gives a hint when you ask and reviews your design when you are ready.
What do interviewers look for?
In any data pipeline design round.
- Clarifies sources, consumers and freshness needs
- Chooses batch, streaming or both with reasons
- Handles late, duplicate and malformed records
- Makes each stage safe to retry
- Explains storage formats and partitioning
- Plans monitoring, backfills and data quality checks
Questions people ask about this interview question
- Which design type is this question?
- It is a data pipeline question. You design GPU telemetry analytics with NL-to-SQL and explain your choices the way you would in an interview.
- Is this question asked at entry level?
- It is marked Hard. It suits engineers preparing for senior interviews, where you are expected to drive the discussion and go deep on trade-offs.
- What should I cover first?
- Start with the data sources, the consumers and how fresh the data must be. Agree on scope with the interviewer before you design anything.
- Can I practise it for free?
- It is a premium question. Sign up and unlock it once for 10 credits; it then stays unlocked. When you finish, Lala, your personalized AI mentor, can review your design for 30 credits.
Related questions
Updated · Written by the SevyDevy team