ML System Design Interview Framework: Six Steps

You end up defending a model before anyone knows what it should predict, how to measure it or where its training answers come from. In an ML system design interview, those gaps leave the rest of your answer with nothing to stand on.

Picture a common setup. A backend developer with three years of experience practises with a friend, who asks them to design search ranking for an online shop. The setup is made up, but the mistake is real.

Within two minutes the developer says they would train a deep neural network, a model with many layers, on click data. The friend then asks three questions. How would you know the new ranking is better? Which clicks count as good? What about a product added yesterday?

None of those questions is about the model. They are about the goal, the metric, the labels and the features. This guide turns them into six steps you can use in any ML design round.

The ML round looks like a classic system design round, but the weight sits elsewhere.

• A classic round asks how servers and databases fit together. An ML round first asks what you predict, and how you know it is good.
• A classic system's answer can be checked request by request. An ML system will be wrong on some requests however good it is, so you judge it by metrics over many requests.
• Labels need their own design. Where they come from and how they are biased can decide the whole answer.
• You still need serving basics, such as latency (how long one request takes) and a fallback (a simple backup answer when the model fails).

An ML system design interview is a round where you design a product that learns from data, such as search ranking or fraud checks. You talk through the goal, metrics, data, model, serving and monitoring, usually on a whiteboard.

Expect discussion and drawing more than coding. Length and format differ by company, so ask your recruiter. This guide plans for a 45-minute round and shows how to stretch it to 60.

For servers, queues and databases on their own, read a classic system design guide first. Rounds about apps built on large language models follow a different shape, covered in our GenAI system design interview guide.

The six steps are: frame the problem, choose metrics, find data and labels, design features, pick a model starting with a baseline, and plan serving and monitoring. Doing them in this order stops you from building a model before you know what it is for.

Think of a family planning a road trip from Pune to Goa. Agreeing why they are going is framing. Deciding what a good trip means, such as arriving before dark, is choosing metrics.

Checking last year's trip notes is the data. Reading road conditions and weather is the features. Picking the route and the car is the model, and watching the road while driving is monitoring.

Where the picture breaks: a trip ends, but an ML system keeps running. Its own choices also change the data it learns from next, which no road trip does.

The diagram shows the order. Notice the last arrow: what you learn in production shapes the next version. If an interviewer jumps ahead, answer briefly and return to your order.

You frame the problem by turning a vague business goal into one clear prediction task. Then you pick metrics that show whether the task is solved, both on old data and with real users.

Step 1: frame the problem

The question will sound vague on purpose, such as improving search. First ask five or six questions that shape the design.

• Goal: what should improve? More purchases, more revenue, or fewer searches that end with nothing found?
• Users: who searches, on which devices, and in which languages?
• Scale: how many searches a day, how many products, and how fast must results appear?
• Today: how does search rank results now? That becomes your baseline.
• Limits: any rules on fairness to sellers, sponsored items or user privacy?

Say the interviewer answers like this. The shop has 40 lakh (4 million) products and 1.2 crore (12 million) searches a day, with peaks of 600 a second. Ranking must finish within 100 milliseconds.

Check the scale out loud: 12 million searches over 86,400 seconds is about 140 a second on average. A peak of 600 is about four times that, so size the system for the peak, not the average.

Today the shop sorts by text match, then by past sales. Now state the ML task in one sentence. Given a query and a shopper, order the matching products so the ones most likely to be bought come first.

This is learning to rank: training a model to put items in the best order. Its scores matter for the order they create, not as answers on their own. Write the task at the top of the whiteboard, so every later choice can point back to it.

Step 2: choose offline and online metrics

You need two kinds of metric. An offline metric is computed on saved past data before launch. An online metric is measured on live shoppers, usually through an A/B test, where random groups see the old or new ranking.

Back to the road trip. The map app's predicted drive time is like an offline metric: quick to check before you leave. Arriving happy and before dark is the online metric, and a good estimate does not promise it.

For ranking, a common offline metric is NDCG at 10, short for normalised discounted cumulative gain. It rewards putting the best products near the top of the first ten. A result counts less the lower it sits, and a perfect order scores 1.

Here is NDCG in a few lines of Python. Each number in the list is a grade for one shown product, in the order the model ranked them. A higher grade means the shopper did more with it.

import math

def dcg_at_k(grades, k):
    # grades are in the order the model ranked the products
    return sum(g / math.log2(pos + 2) for pos, g in enumerate(grades[:k]))

def ndcg_at_k(grades, k=10):
    ideal = dcg_at_k(sorted(grades, reverse=True), k)
    return dcg_at_k(grades, k) / ideal if ideal > 0 else 0.0

# One search for "steel water bottle": 3 bought, 2 carted, 1 clicked, 0 skipped
print(round(ndcg_at_k([0, 3, 1, 0, 2], k=5), 2))   # prints 0.66

Run it with python3 and it prints 0.66. The log2(pos + 2) part is the discount: the first result divides by 1, the second by about 1.585, the third by 2.

The ideal line sorts the same grades into the best order, so dividing by it gives a score from 0 to 1. Here a skipped product sat above the bought one, so the score is only 0.66.

This version uses the grade itself as the gain, with the same log2 discount as the textbook. The common textbook version uses 2 to the power of the grade, minus 1, which rewards top grades more steeply. With that gain the same list scores 0.65, so say which version you mean.

Online, pick purchases per search as the main metric. Then add guardrails: numbers that must not get worse even if the main metric improves. Good ones here are slow searches, searches with zero results, and return rates for products bought from search.

Labels usually come from what users did after past searches, such as clicks and purchases. Features are the signals about the query, product, shopper and moment that the model reads to make each prediction.

Step 3: data and labels

A label is the right answer for one training example. Paying people to grade results for 40 lakh products is slow and costly. So use implicit labels, taken from behaviour instead of human grading.

They come from event logs, which are records of what shoppers saw and did. Grade each product shown after a search: a purchase gets 3, add to cart gets 2, a click gets 1, and shown but skipped gets 0.

Clicks alone are noisy, because a catchy photo can earn clicks without sales. Purchases are rarer but closer to the goal, which is why the grades mix both.

Logs carry a trap called position bias: top products get more clicks partly because people see them first. A 2016 learning-to-rank paper states that using raw clicks directly as training data gives sub-optimal rankings.

Its fix weights each click by one over the chance that the click's position was looked at, so a click far down the list counts for more. The engineer section below shows this with a short simulation.

In an interview, two simpler fixes are often enough. The first is to train with position as a feature, then set it to one default value for every product at serving time.

Rules of Machine Learning, a well-known engineering guide by Martin Zinkevich, recommends exactly this and adds one condition. Keep the position feature separate from the other features, so the model does not mix it into everything else.

The second fix is to shuffle results for a small slice of traffic. Then you see how often each slot gets clicked when products land there at random.

The next trap is data leakage: information in training that would not exist at prediction time. Suppose a feature counts a product's total sales, including the test week. The model then sees the answer early, and offline scores look far too good.

The fix is to split by time, not at random, so you train on earlier days and test on later ones. The same engineering rules recommend testing on data gathered after the training data. The code below turns logged events into grades, then splits by day.

GRADE = {"purchase": 3, "add_to_cart": 2, "click": 1}

def grade_for(events):
    # events: everything one shopper did with one product after one search
    return max((GRADE.get(e, 0) for e in events), default=0)

def split_by_day(rows, cutoff_day):
    train = [r for r in rows if r["day"] < cutoff_day]
    test = [r for r in rows if r["day"] >= cutoff_day]
    return train, test

print(grade_for(["click", "add_to_cart", "purchase"]))   # prints 3
print(grade_for([]))                                     # prints 0
rows = [{"day": d} for d in range(1, 31)]
train, test = split_by_day(rows, cutoff_day=25)
print(len(train), len(test))                             # prints 24 6

Run it and it prints 3, then 0, then 24 6. The max line keeps the strongest action, so a click, a cart add and a purchase together count as 3, not 6.

The split keeps every test day after every training day: days 1 to 24 train the model and days 25 to 30 test it. That matches how the model will meet the future in production.

Step 4: features

A feature is one number or category the model reads, such as a product's price. Group your features so the interviewer can see you covered every side of the problem.

• Query: length, detected category, language, and whether it names a brand. Example: steel water bottle 1 litre.
• Product: price, rating, stock, seller rating, and sales in the last 7 days.
• Query and product together: how well the title matches the query, and whether categories agree.
• Shopper: past categories, usual price range, and city. Use only what your privacy rules allow.
• Context: device, time of day, and whether a sale is running.

Then name training-serving skew. This is when a feature is computed one way for training and another way live, say with a different rounding rule. The model then meets numbers it never learned from.

Two fixes come from the same engineering rules. Re-use one piece of code for features in training and serving, and log the features used at serving time so you can train on them later.

A feature store, a shared service that computes features once and serves them to both training and the live system, is one common way to do the first.

Propose a simple baseline first, then a learned ranker in two stages. A fast first stage finds a few hundred candidates, and a slower model puts them in order.

The baseline here is today's ranking: text match, then past sales. It gives you a number to beat and tests your logs and metrics before any model exists. It also becomes the fallback.

The same engineering rules make this point bluntly: the first model gives the biggest boost, so it does not need to be fancy. Use it to get the infrastructure right.

Next comes the two-stage design. Candidate retrieval is a fast search that cuts 40 lakh products down to about 500 that match the query. It combines keyword search with search by meaning, which finds similar products even when the words differ.

The ranking model then scores only those 500 and shows the top 40. The cache in the diagram holds candidate lists for popular queries, not final orders, because the final order depends on the shopper.

For the ranker, propose gradient-boosted trees. A decision tree is a chain of yes or no questions about features, and boosting adds many small trees, each fixing the errors of the ones before.

They suit number-like features such as price and match scores, and they score fast. In our test on made-up data, 300 trees reading 30 features scored 500 products in a median of about 10 milliseconds on one CPU core.

That figure covers the model alone, not feature lookups or the network, so measure your own. It suggests model scoring fits well inside a 100 millisecond budget on similar hardware.

Also say how the ranker learns. Pointwise training scores each product alone. Pairwise training learns which of two products ranks higher, and listwise training looks at the whole list for a query.

A 2009 survey of learning to rank reported that results on the public test sets it studied suggest listwise methods work best. It called pointwise training possibly sub-optimal, so naming a pairwise or listwise loss is a safe interview answer.

Serving means running the model inside its time budget, with a fallback if it fails. Monitoring means watching inputs, outputs and business numbers every day, so you notice when the model goes stale.

Start with the budget: about 100 milliseconds for 500 products. Product features are computed ahead of time, and only a few, like the match score, are computed per query. If the ranker times out, the service shows the baseline order instead of an error.

Next, the rollout. First run the new model in shadow mode: it scores every search, but shoppers still see the old ranking. Then comes an A/B test on 5% of shoppers, and a wider rollout only if the metrics hold.

A 2020 paper on models in production groups monitoring into three areas. Track model performance and input data, detect outliers (unusual inputs) and data drift, and explain past predictions.

Data drift means live inputs stop looking like the training data, as with festive-season searches. Our guide on model drift covers the difference between drift in inputs and drift in what the right answer is.

There is one more risk. A 2015 paper on hidden technical debt in ML systems calls it a direct feedback loop: a model can influence the selection of its own future training data.

Here the ranker decides what shoppers see, shoppers click what they see, and tomorrow's labels come from those clicks. The same paper says some amount of randomisation can soften such loops. The shuffled slice from step 3 is exactly that.

In the diagram, follow the Yes arrow. A retrained model passes two checks before it reaches every shopper.

Give about 10 minutes to framing and metrics, 14 to data, labels and features, 8 to the model, 8 to serving and monitoring, and the last 5 to trade-offs.

Other plans exist. One public interview-prep framework, as of October 2026, gives 5 to 7 minutes to framing and 2 to 3 to a high-level design. Data and modelling get about 10 each, and inference and evaluation about 7. The rest goes to deep dives the interviewer steers.

The shared idea matters more than the exact minutes: frame first, keep a clock, and leave room for follow-ups. If the interviewer pushes hard on labels, follow them and shorten something else. With 60 minutes, spend the extra time on data and monitoring.

The costliest mistake is jumping to the model before the goal and metrics are clear. Most other mistakes are forms of skipping a step.

• Naming a model in minute two. Fix: say the ML task and the metric first.
• One metric only. Fix: pair offline with online, and add guardrails.
• Labels taken for granted. Fix: say where each label comes from and how it is biased.
• A random split on time-based data. Fix: split by time and check for leakage.
• No baseline. Fix: name today's system as the number to beat.
• Stopping at the trained model. Fix: cover rollout, drift and retraining.
• Long silences. Fix: think out loud, for example that you are choosing between two label designs, and why.

Companies use their own rubrics and rarely publish them. A useful way to practise is to score yourself on the same six areas this guide uses, from weak to strong.

Here is a sample practice rubric built from the six steps. It is not any company's sheet; it shows what strong looks like in each area.

• Framing: asks focused questions and turns the goal into one ML task with clear input and output.
• Metrics: links offline and online metrics, adds guardrails, and knows they can disagree.
• Data and labels: names label sources, their biases, and a split that avoids leakage.
• Modelling: starts with a baseline and justifies each step up in cost and gain.
• Production: covers latency, fallback, rollout, monitoring and retraining.
• Communication: keeps time, draws clearly, and explains trade-offs without being asked.

Production gets a full row for a reason. A 2020 survey of published deployment case studies found that practitioners hit issues at every stage of the deployment workflow, not only in modelling.

The public prep framework mentioned earlier lists green flags and red flags for each section. It stresses that interviewers are assessing how you approach the problem.

Simulate two products whose real appeal you know, put one on top, and count clicks. In this simulation, raw clicks crown the top product even though the other is three times more appealing. That happens whenever the lower slot is seen less than a third as often.

In the code, product A sits in slot 1 and is always seen. Product B sits in slot 3, which shoppers look at 30% of the time. Once seen, B gets clicked three times as often as A. All numbers are made up.

Pause and predict: B is three times as appealing, but A is always on top. Which product collects more raw clicks over 100,000 searches?

import random

random.seed(7)
SEE = {1: 1.0, 3: 0.3}                  # chance a shopper looks at slot 1 or slot 3
APPEAL = {"A": 0.10, "B": 0.30}         # chance of a click once a product is seen
SLOT = {"A": 1, "B": 3}                 # the old ranking always put A on top

clicks = {"A": 0, "B": 0}
weighted = {"A": 0.0, "B": 0.0}
for _ in range(100_000):                # 100,000 searches
    for p in ("A", "B"):
        seen = random.random() < SEE[SLOT[p]]
        if seen and random.random() < APPEAL[p]:
            clicks[p] += 1
            weighted[p] += 1 / SEE[SLOT[p]]   # inverse of the chance it was seen

print("raw clicks:", clicks)
print("weighted:", {p: round(w) for p, w in weighted.items()})

Run it with python3. It prints raw clicks: A 9912 and B 9164, then weighted: A 9912 and B 30547. The exact counts come from the fixed seed; across 20 different seeds, A won on raw clicks every time.

What just happened? Raw clicks say A is the better product, so a model trained on them would keep A on top. Weighting each click by one over the chance it was seen recovers the true ratio of about 3 to 1.

The catch is that you rarely know the chance of being seen. The 2016 paper estimated it with a randomised swap: for some searches, swap the top result with the one at a randomly chosen rank, then compare click rates.

That is a real cost, because some shoppers see worse results. Say so in the interview, and size the slice small.

Most follow-ups test one of the six steps, not the model. A strong first answer names the likely cause and the first thing you would check.

The question about a product added yesterday is the cold-start problem: items with no history yet. It tests features, because the answer is signals that exist from day one.

Skew comes first in the offline-up, online-down question because it is cheap to test. Run the same example through training and serving code; the same engineering rules say a mismatch there usually points to an engineering error.

Name a few numbers, a time window and an action for each. Saying you will monitor drift is vague; saying what fires an alert, and what happens next, shows you have run a system.

The thresholds below are made up for the running example. Set yours from a few weeks of your own history.

• Missing features: more than 1% of searches with a missing feature value for 15 minutes. Action: page the on-call engineer.
• Latency: p99 ranking time, the time 99% of requests beat, above 100 milliseconds for 5 minutes. Action: serve the baseline order automatically.
• Fallback rate: more than 2% of searches served the baseline order in an hour. Action: alert, because the model is quietly not running.
• Zero-result searches: up 20% against the same hour last week. Action: check retrieval before the ranker.
• Predicted versus actual: if the model outputs purchase probabilities, predicted rate off the observed rate by over 10% in any category for a day. Action: investigate before retraining.

The last check comes from the 2015 hidden technical debt paper. It calls comparing predicted and observed label rates a surprisingly useful diagnostic, though by no means a full test.

It needs scores that are probabilities, such as a pointwise model trained on purchase or no purchase, with its calibration checked. Pairwise and listwise scores only set an order, so pair them with a separate purchase-probability model if you want this check.

The same paper suggests action limits, sanity bounds on what the system may do. For search, one could be that no single seller fills more than half of the top 10 results.

You walk the six steps on one concrete problem, keep the clock, and answer follow-ups as they come. Here is the full 45-minute answer for the running example.

1. 1

   Frame (minutes 0 to 5)

   Ask about goal, scale, latency and today's ranking. Write the task: given a query and a shopper, order matching products by chance of purchase.

2. 2

   Metrics (5 to 10)

   Offline: NDCG at 10 on graded labels. Online: purchases per search in an A/B test. Guardrails: slow searches, zero-result searches and return rates.

3. 3

   Data and labels (10 to 17)

   Thirty days of event logs, graded 3, 2, 1 or 0. Name position bias and its two fixes, then split by time to avoid leakage.

4. 4

   Features (17 to 24)

   Five feature groups, from query to context. Name training-serving skew, shared feature code and logged serving features.

5. 5

   Model (24 to 32)

   Baseline first, then retrieval of 500 candidates and a boosted-tree ranker with a pairwise or listwise loss. Answer cold start with history-free features.

6. 6

   Serving and monitoring (32 to 40)

   The 100 ms budget, the baseline fallback, shadow mode, a 5% A/B test, alert thresholds and a retraining trigger.

7. 7

   Wrap-up (40 to 45)

   Name the biggest risk, the feedback loop, and one next step, such as personalising by shopper history once the basics hold.

Notice that the model gets 8 of 45 minutes. The rest is what makes the model worth building, and it is where most follow-up questions land.