Real-Time Features: Batch vs Streaming vs Request-Time

The model gives confident answers about a world that has already changed. Nothing crashes, and every number it reads was correct when it was computed. It is just too old for the question being asked. Real time features fix this by keeping fast-changing inputs only seconds or minutes old.

Picture a common setup. A quick-commerce grocery app promises deliveries in about 15 minutes from 18 dark stores, small warehouses that only serve online orders. It handles about 40,000 orders a day. The setup is made up, but the problem is real.

Two machine learning models drive the app. The ETA (estimated time of arrival) model predicts how long an order will take, and the app shows that number before checkout. The recommendation model picks the "You may also need" row on the cart screen.

The team builds every model input with one nightly job. The job reads the order database and writes each value into a fast lookup store by 4 a.m.

Then comes a rainy Friday in the festival season. Orders at the busiest store triple between 7 and 8 p.m. The app keeps promising 14 minutes, and real deliveries take 35. Support gets 600 angry chats in one evening.

• The ETA model's "orders in the last hour" input still holds last night's number, so the model thinks the store is quiet.
• Its "free riders near the store" input is a daily average, so it cannot see that most riders are stuck in the rain.
• The recommendation row keeps suggesting fresh coriander after the store sold out at 6:40 p.m.

This guide follows the team as it decides which inputs need to be fresh, and what that freshness costs.

Real time features are model inputs computed from fresh data, usually seconds to a few minutes old, so the model sees what is happening now. Batch features, the opposite, are rebuilt every few hours or days.

For the ETA model, one feature is the distance from the store to the customer. Another is how many orders the store received in the last ten minutes. A third is the store's average packing time over the past week.

Those three features change at very different speeds. Packing time drifts over days. The ten-minute order count can double in a quarter of an hour. Distance is different for every order, so nobody can compute it before the order exists.

Instacart's engineering team described the same split for grocery delivery in a 2022 post. Its batch features change rarely, such as a product's category or nutrition details. Its real-time features include item availability, the number of shoppers online and the number of incoming orders.

Two more words help. Freshness is how recent the data behind a feature value is. Staleness is the opposite: the age of that data when the model reads it. If the order count was last updated 90 seconds ago, its staleness is 90 seconds.

The difference is when the value is computed. Batch features are computed on a schedule, streaming features are updated as each event arrives, and request-time features are computed at the moment of the prediction.

Think of a sugarcane juice stall. The owner can fill bottles at 6 a.m. and keep them in an ice box. Those bottles are cheap to serve and always ready, but by evening they are not really fresh. That is a batch feature.

He can also press a fresh glass for each customer. It is as fresh as juice gets, but every customer waits for the machine, and he cannot serve a big crowd fast. That is a request-time feature.

In between, his helper presses a jug every few minutes and keeps topping it up. Each glass poured from the jug is only minutes old, and nobody waits for the press. But the helper is paid all day, even in slow hours. That is a streaming feature.

Where the picture breaks: old juice is worse for every customer, but an old feature value hurts only if the real value has changed since. A week-old packing time can be perfectly good, because packing time barely moves.

Batch features

A batch feature is computed by a scheduled job that reads stored history, often from a data warehouse, a database built for big analytical queries. The job writes results into an online store, a fast key-value database the model reads at prediction time. Weekly packing time per store is a good batch feature.

Streaming features

A streaming feature is updated continuously by a stream processor, a program that reads events one by one and keeps running totals. An event here is a small record such as "order placed at store 42 at 7:02 p.m." The processor writes each new total to the online store.

Request-time features

A request-time feature, also called an on-demand feature, is computed inside the prediction request from data that arrives with it. Distance to the customer and the items now in the cart are examples. Nothing can be prepared in advance, because the inputs did not exist a second ago.

Chip Huyen, who writes widely about machine learning systems, draws the same line in a 2022 post. Features computed at prediction time are fresh, but their computing time adds directly to the user's wait. Features computed ahead from streams avoid that wait, but can be seconds old.

Watch the names, because they differ between sources. Huyen calls prediction-time features "real-time features" and stream-computed ones "near real-time features". Feast, an open-source feature store (a system that computes, stores and serves model inputs), builds request-time features with on demand feature views. Its docs label them beta as of 19 October 2026.

Here is how all three reach the ETA model. Follow each path into the online store and the model server.

The table below puts the three side by side for the delivery app. Read the cost and delay rows closely, because they drive every decision later in this guide. The rupee figures are made up, like the rest of the setup.

Batch features are computed by a scheduled job over stored history, then copied into a fast online store so the model can read them in a few milliseconds. This is the cheapest and simplest kind, and most features should stay here.

The team's nightly job is a few SQL queries, written in the standard language for asking a database for data. One computes each store's average packing time over seven days. Another computes each customer's favourite categories from 90 days of orders. A third computes how long riders usually wait at each apartment complex gate.

Batch work is cheap because it runs once and then stops. It reads the data in big, efficient scans, and the machines shut down when it finishes. If the job fails, the team can rerun it the next morning, and the old values keep working meanwhile.

The price is staleness. A value written at 4 a.m. is 15 hours old at 7 p.m. For packing time, that does not matter, because it barely moves in a day. For an order count, it is useless.

Streaming features stay fresh because a stream processor updates them every time a new event arrives, instead of waiting for a nightly run. The model then reads a value that is usually only seconds old.

In the delivery app, the customer app and the rider phones publish events to Kafka, an open-source system that stores streams of events. Kafka groups events into topics, such as one topic for orders and one for rider status. It keeps order within each partition, a slice of a topic, not across a whole topic. A stream processor reads the topics and updates counts per store.

Instacart's 2022 post describes the same shape. Raw events go to Kafka, Flink (an open-source stream processor) turns them into features, and the features land in a feature store. The post notes that routing events through Kafka adds some delay, usually within a few hundred milliseconds.

Counting over a window

Most streaming features are counts or sums over a window, a slice of time such as the last ten minutes. The Apache Flink 2.3 documentation lists four built-in window types. Tumbling windows have a fixed size and never overlap, and session windows close after a gap with no activity.

Sliding windows also have a fixed size, but a new one starts every slide, such as every minute. They overlap when the slide is shorter than the size. Flink also offers a global window, which on an endless stream needs your own rule for when to fire.

Here is the idea in plain Python for one store's "orders in the last 10 minutes" feature. A real stream processor adds state backup and many machines, but the counting logic looks like this. Times are in seconds.

from collections import deque

WINDOW_SECONDS = 600  # the last 10 minutes

class RecentOrders:
    def __init__(self):
        self.times = deque()  # order times, oldest first

    def add(self, event_time):
        self.times.append(event_time)

    def count(self, now):
        cutoff = now - WINDOW_SECONDS
        while self.times and self.times[0] <= cutoff:
            self.times.popleft()  # forget orders older than the window
        return len(self.times)

recent = RecentOrders()
for t in [0, 100, 590, 650]:  # order times, in seconds
    recent.add(t)
print(recent.count(now=700))  # prints 2: only 590 and 650 are in the window

recent.add(50)  # a late event: this order happened at second 50
print(recent.count(now=700))  # prints 3, but the true count is still 2

The deque is a list that is cheap to change at both ends. New orders join on the right, and the while loop drops old ones from the left. Every few seconds, the processor calls count and writes the result to the online store.

Run it and the first print shows 2. Pause and predict before reading on: why does the second print show 3 after one late event?

What just happened? The code assumes orders arrive in time order. The late order from second 50 sits behind the order from second 650, and the loop only checks the left end. So an order that is outside the window still gets counted.

Why streaming costs more

Streaming costs more because the machines run all day, even at 3 a.m. when almost nobody orders. Someone must also watch the pipeline, because a stuck processor quietly freezes every feature it writes.

Back at the juice stall, the helper with the jug is paid for the whole day. On a slow afternoon, he still presses juice that may sit unsold. That is the streaming bill: you pay to stay ready.

Compute a feature at request time when its inputs only exist inside the request, and the computation is quick and unlikely to fail. Distance to the customer and the contents of the current cart are classic examples.

Request-time work has a hard limit: it adds to the time the customer waits. Say the checkout screen must show the ETA within 150 milliseconds, and the model itself takes about 30 of those (made-up numbers). The other 120 milliseconds also pay for the network and store lookups, so each request-time feature gets only a small slice.

• Good request-time features: distance from store to address, number of items in the cart, whether the cart holds frozen items, the hour of the day.
• Poor request-time features: anything that scans history, calls a slow outside service, or needs a big database query.
• Mixed features: combine a stored value with a request value, such as stored free riders together with this order's distance.

The recommendation model shows why request-time features matter. A customer who just added poha and peanuts probably needs curry leaves next. Last night's batch job could not know that, because the cart did not exist then.

A feature needs to be as fresh as the point where extra freshness stops improving the model's predictions. You find that point with a staleness test, not with a guess.

A staleness test replays past data, but feeds the model feature values that are deliberately old. The team uses four delays: one minute, ten minutes, one hour and one day. For each delay, it measures the ETA model's average error in minutes.

Replay is possible because Kafka keeps events after they are read. Its documentation says events are not deleted after consumption, and each topic has its own setting for how long events are kept. So the team can rebuild a feature's value at any past moment within that period.

Check that period before you plan a replay. In Kafka 4.3, the broker default is log.retention.hours of 168, which is seven days. A four-week test needs a longer setting or a copy of the events in the warehouse.

1. 1

   Pick a busy test period

   Take four weeks of orders that include rainy evenings and festival days, when freshness should matter most.

2. 2

   Rebuild each feature with a delay

   For every order, look up what the feature would have said one minute, ten minutes, one hour and one day earlier.

3. 3

   Run the model on each version

   Keep the model the same and change only the delay, so any change in error comes from staleness alone.

4. 4

   Compare errors at peak hours

   An average over the whole day hides the problem. Look at 6 to 10 p.m. separately.

5. 5

   Price the freshness

   For each feature, set the error saved next to the extra cost of making it fresher, and pick the cheapest level that is good enough.

The results are clear, and like every number in the setup they are made up. With the order count one day stale, the peak-hour ETA error is 9 minutes. At one hour stale, it is 6 minutes. At one minute stale, it drops to 3 minutes, and ten minutes gives nearly the same.

Weekly packing time tells a different story. Making it fresher than one day changes the error by less than 10 seconds. So it stays in batch, and the money goes to the order count instead.

Now the cost side. Streaming the order count and free riders costs about Rs 3,600 a day. Support estimates each wrong-ETA complaint at about Rs 40 in refunds and staff time. So the streaming pays for itself once it prevents 90 complaints a day (3,600 divided by 40).

Use the diagram below to place any new feature. Start at the top question and answer honestly.

The app ends up with most features in batch, three streaming features and a few cheap request-time features. Freshness is spent only where the staleness test showed a real gain.

• Streaming: orders in the last 10 minutes per store, free riders within 2 km of each store, and items sold out per store.
• Request-time: distance to the address, items in the cart, the hour and the day of the week.
• Batch: weekly packing time, gate waiting time per apartment complex, customer category preferences, and items often bought together.

The sold-out feature fixes the coriander problem. When the last bunch sells, an event reaches the recommendation model within seconds, and the item leaves the row. Instacart's 2022 post reports a similar change: item availability scores updated in seconds instead of every couple of hours.

On the next rainy Friday, the order count at the busiest store climbs at 7 p.m. and the ETA climbs with it, to 28 minutes. Some customers still wait. Fewer complain, because the promise matches the truth.

Real-time features fail in quieter ways than batch ones. Values freeze when a pipeline stalls, late events give wrong counts, and training data stops matching what the model sees live.

Late events and event time

Rider phones lose signal in basements and send their events late. Flink's documentation separates processing time, the clock of the machine doing the work, from event time, when the event actually happened on its device. Counting by event time puts a late "order picked" event into the right window, as long as it arrives before that window closes.

But how long should the processor wait for stragglers? Flink uses a watermark, a marker in the stream that says event time has reached a certain point. Elements that arrive after the watermark has passed their timestamp are called late elements.

In Flink 2.3 event-time windows, late elements are dropped by default once the watermark passes the end of their window. You can set an allowed lateness, so they still update the window. The default event-time trigger, the rule that decides when a window outputs its result, then fires it again. You can also send them to a side output, a separate stream of late data.

Frozen values

If the stream processor stops, the online store keeps its last values. The model keeps reading an order count from 40 minutes ago, with no error anywhere. This is worse than a crash, because nobody notices.

The team stores a written-at time next to every streaming value. The online store is Redis, an in-memory key-value database, and each key also gets an expiry. Redis documentation says a key is deleted automatically after its timeout, and that SET and other overwrites clear the timeout.

That second detail is easy to miss. A writer that sets the expiry once and then overwrites with a plain SET leaves keys with no expiry at all, so frozen values never disappear. The fix is to set the expiry in the same command on every write.

You can check this on your own Redis with redis-cli. TTL reports the seconds left before a key expires, and -1 means the key has no expiry. This output is from Redis 7.0.15.

$ redis-cli SET feat:store42:orders_10m 37 EX 120
OK
$ redis-cli TTL feat:store42:orders_10m
(integer) 120
$ redis-cli SET feat:store42:orders_10m 41
OK
$ redis-cli TTL feat:store42:orders_10m
(integer) -1

The first SET writes the value and a 120-second expiry in one command. The second, plain SET keeps the key but removes its expiry, so TTL drops to -1. If you must keep an existing expiry, Redis 6.0 and later accept SET with the KEEPTTL option.

On the reading side, the model server checks each feature's age before using it. In the code below, online_store.get is a made-up helper that returns a value and the time it was written. FALLBACK holds safe default values, such as each store's usual evening order count.

import time

MAX_AGE = {"orders_10m": 120, "free_riders": 60, "avg_pack_7d": 2 * 86400}

def eta_features(store_id, distance_km):
    features = {"distance_km": distance_km}  # request-time feature
    for name, max_age in MAX_AGE.items():
        value, written_at = online_store.get(store_id, name)
        stale = value is None or time.time() - written_at > max_age
        if stale:
            value = FALLBACK[name][store_id]  # safe default, never a crash
        features[name] = value
        features[name + "_is_stale"] = int(stale)
    return features

MAX_AGE is the freshness promise for each feature, in seconds: two minutes for orders, one minute for riders, two days for packing time. An expired key comes back as None, so it also takes the fallback path.

The stale flag goes into the features too. So the model can tell a real value from a fallback, as long as its training data also contains these flags and fallbacks. With a stand-in store, a 30-second-old order count passed through and a 400-second-old rider count was replaced and flagged.

Training data that does not match live data

Training-serving skew means the model sees different feature values in training than in live use. Streaming makes this easy to cause, because the live count comes from stream code while training data often comes from a separate SQL query.

Here is one such bug in the setup. The training SQL counted cancelled orders, but the stream code skipped them. The model learned from one definition and ran on another, and its peak-hour error crept up.

Huyen describes a guard called log and wait. The feature values computed for each live prediction are logged, then reused when the model is retrained. Joined with the real outcome, such as the actual delivery time, those logs show exactly what the model saw.

The diagram below shows that loop for the ETA model. Follow the logged features from the model server into the training data.

A feature store can help too, because one definition can feed both paths. For example, Feast documents that an on demand feature view runs the same transformation for training data and for live lookups. Feature stores are a large topic of their own.

Explain them as a freshness budget: list the features, say how fresh each must be and why, then show the pipeline and the fallbacks for each level.

1. 1

   Name the prediction and its deadline

   For example: predict delivery time at checkout, within 150 milliseconds. The deadline limits how much request-time work you can do.

2. 2

   List features with their freshness

   Write each feature with a tag: batch, streaming or request-time. Give a one-line reason for every streaming one.

3. 3

   Draw the three paths

   A nightly job into the online store, events through a stream processor into the same store, and request-time code inside the model server.

4. 4

   Explain how you chose

   Describe the staleness test and the cost comparison. This shows you spend money on freshness only where it pays.

5. 5

   Cover failure and skew

   Maximum age, fallback values, stale flags, alerts on feature age, event time with watermarks for late events, and logged features for training.