Design an OTT Streaming Platform for Live Sport

The stream freezes, or the app never gets past its loading spinner, because the parts that let people in were sized for a normal evening. OTT system design for live sport is mostly about that rush. The video itself is the easier half.

Picture a common setup. A sports streaming app in India shows live cricket and kabaddi on phones and TVs. The setup is made up, but the problem is real.

On a normal evening, about 1.5 lakh people watch at once. Last season's state T20 league semi-final was different. At 6:55 pm, 6 lakh people were watching the pre-match show, the toss was at 7:00, and by 7:08, 24 lakh were watching.

That is 18 lakh new viewers in eight minutes, or about 3,750 people pressing play every second. The team had planned for concurrency, the number of people watching at the same time. Nobody had planned for joins per second.

• At 7:02, the entitlement service started timing out. It answers one question: has this person paid for this match? On a normal evening it sees about 300 new viewers a second.
• The app retried each failed check three times, straight away. Every timeout became four requests, so the struggling service got even more traffic.
• In the last over, viewers peaked at 31 lakh and something else broke. The CDN had been told never to cache live playlists, so every reload went to the origin servers, and video froze in two regions for about 90 seconds.

The final is six weeks away and should draw a bigger crowd. This guide follows what the team changes, and why each change matters in the two worst minutes of the night.

OTT system design is planning how a video app gets content from its source to millions of screens. It covers encoding, packaging, storing, delivering and playing video, plus the login and payment checks around it. For live sport, the hardest part is serving the same few seconds of video to everyone at once.

A film library is easier in one way. Viewers spread out over the whole day and over thousands of titles. Live sport puts everyone on one title at one moment, and the size of the crowd follows the match. A rain delay empties the app, and a close finish fills it.

You need six words for the rest of this guide:

• Bitrate: how much data the video uses per second. A 3 Mbps stream (3 million bits a second) looks sharper than a 0.8 Mbps one, but needs a faster network.
• Segment: a short video file holding a few seconds of the match, such as 6 seconds. A live stream is a long chain of these files.
• Playlist: a small text file listing the latest segments. The player reloads it again and again to learn which new segment to fetch.
• Origin: the servers that hold the master copy of every segment and playlist.
• CDN: a content delivery network, a large set of cache servers in many cities. Each one keeps copies near viewers, so the origin is not asked every time.
• Edge: one of those CDN cache servers, the one your phone actually talks to.

Picture a city marathon to hold the two problems in your head. The water stations are stocked before the gun, because no truck can drive through the runners later. Then the pack arrives, and thousands of hands reach for cups in the same minute.

• Stocking the stations before the gun is adding servers before the toss.
• The pack reaching a station together is every player asking for the same new segment in the same few seconds.
• One table of cups at each station, instead of one central tap, is the CDN edge near each crowd.

Where the picture breaks: a cup is used up once, but a cached segment can be served again and again. That is why caching works so well for video and not at all for personal checks like login.

Live video moves through a pipeline. The stadium feed is compressed, turned into several qualities, cut into segments, stored at the origin and copied out through CDN edges to each player. Every step adds a little delay.

An encoder at the ground compresses the broadcast feed and sends it to the cloud. A transcoder then makes five versions of it, from a small 240p picture (240 lines of pixels tall) to full 1080p. A packager cuts each version into 6-second segments and writes a playlist for each. The origin stores all of it, and the CDN carries it to viewers.

Follow the video from the stadium feed at the top of the diagram. Then notice the last arrow, from the app to login and entitlement, which never touches the CDN.

The origin shield is a middle layer of cache between the edges and the origin. Without it, every edge that misses asks the origin itself. With it, edge misses go to the shield first. Amazon CloudFront's Origin Shield, for example, merges requests for the same file, so as few as one reaches the origin.

The two paths behave very differently under load. The video path is the same bytes for everyone, so caches absorb almost all of it. The control path is personal: your login, your subscription, your home screen. A shared cache cannot answer it for you, so it needs its own plan.

In HLS, the server cuts video into short segments and lists them in a playlist, which the player reloads every few seconds to find new ones. HLS, short for HTTP Live Streaming, is described in RFC 8216, a public document published in 2017 in the internet's RFC series. It is informational, not an official internet standard.

There are two kinds of playlist. The top one lists the qualities on offer. RFC 8216 calls it the master playlist, and the newer HLS draft calls it the multivariant playlist. Each quality then has its own media playlist, which lists that quality's latest segments in order.

Every media playlist states a target duration. Each segment's length, rounded to the nearest whole second, must not exceed it. For this app it is 6 seconds. A live playlist has no end marker, so the player keeps reloading it.

How often does it reload? RFC 8216 says that after a reload finds a changed playlist, the player must wait at least the target duration before the next one. If nothing changed, it waits half the target duration and tries again.

Now do the sum for the last over. With 31 lakh viewers, each reloading about once every 6 seconds, that is about 5.2 lakh playlist requests a second. Last season the CDN passed every one of them to the origin, because the playlists were marked as never cacheable.

Segments create a second wave. Every 6 seconds a new segment appears in each of the five qualities. Every player near the live edge asks for it within those 6 seconds, many in the same second. This is the pack reaching the water station.

Why live streams run behind the real match

Live streams lag mostly because the player keeps a safety buffer. RFC 8216 says a player should not start a segment that begins less than three target durations from the end of a live playlist. With 6-second segments, that alone puts a viewer about 18 seconds behind.

Encoding, packaging and delivery add a few more seconds on top. The newer HLS draft adds a low-latency mode built from three tools.

• Partial segments are small pieces of a segment, published at the live edge before the whole segment is finished.
• Blocking playlist reload lets the player ask for the next version of a playlist, and the server holds the request until that version exists.
• Preload hints name the next piece before it exists, so the player can ask early and the server holds the request until it can respond.

The player measures how fast segments are arriving and how much video is buffered, then picks the highest quality that fits, segment by segment. This is called adaptive bitrate streaming. RFC 8216 says clients should switch between versions to adapt to network conditions.

The set of qualities is called a ladder, and each quality is a rung. Here is the app's ladder for the final.

Real players use many rules to choose a rung. This app's player uses a simple one, shown below in Python. It takes the measured network speed in kilobits per second (kbps) and the seconds of video already buffered.

LADDER_KBPS = [400, 800, 1600, 3000, 5000]   # rungs, lowest first

def pick_rung(network_kbps, buffer_s, top_allowed_kbps=5000):
    budget = network_kbps * 0.8               # keep 20% spare
    if buffer_s < 8:                          # buffer running low: play safe
        budget = network_kbps * 0.5
    budget = min(budget, top_allowed_kbps)    # cap set by the server
    fits = [r for r in LADDER_KBPS if r <= budget]
    return fits[-1] if fits else LADDER_KBPS[0]

print(pick_rung(4500, 20))   # 3000
print(pick_rung(4500, 5))    # 1600

What just happened? A phone measuring 4,500 kbps with 20 seconds of buffer gets a budget of 3,600, so it plays the 3,000 rung. With only 5 seconds of buffer, the budget falls to 2,250, and it plays 1,600. Run it and you should see 3000, then 1600.

Two lines matter most. The buffer check drops quality early when the buffer runs low, because a lower picture is better than a frozen one. The top_allowed_kbps cap is the operations team's lever: the app reads it from a small config file, so the team can switch off the 5 Mbps rung during overload.

That cap is worth a lot at the peak. If the 15% of viewers on 5 Mbps drop to 3 Mbps, 31 lakh viewers use about 0.93 Tbps less CDN bandwidth. A terabit a second (Tbps) is a million Mbps.

Spikes hit at the toss and the last over because those are the moments people decide to start watching, and they all decide within minutes. Concurrency climbs fast, but joins per second jump much faster.

The toss is at a known time, so that rush is predictable. The last over is not: nobody knows at 6 pm whether the match will be close at 10:40 pm. A close finish also pulls people back from other apps, often because a friend messaged them.

Your own app can make the spike worse. Last season the app sent a push notification to 50 lakh phones when the semi-final reached its last over. Lakhs of people tapped it within one minute. Now the team spreads such sends over ten minutes.

A join costs far more than a minute of watching. Here is what one tap on Play does:

• Load the home screen, which is personal, so it comes from the app's own API servers, the backend computers that answer the app's requests.
• Refresh the login token, a signed pass that proves who you are.
• Ask the entitlement service whether you may watch this match.
• Fetch the multivariant playlist, then one media playlist, then about three segments to fill the buffer.

Only the last step goes through the CDN. The first three hit the app's own servers. At 3,750 joins a second, that is over 11,000 API requests a second, against about 900 on a normal evening.

Once people are watching, the load moves to bandwidth. With the ladder shares above, the average viewer uses about 2.23 Mbps. At 31 lakh viewers that is about 6.9 Tbps (terabits a second) from the CDN, which only a spread of edges in many cities can serve.

A CDN protects the origin by answering nearly every request from caches. Done well, the origin sees about one request per new file per region, not one per viewer. Three settings decide whether that holds: cache times, request collapsing and serving slightly old copies.

Cache times for each kind of file

Segments never change once written, so they can be cached for hours. Live playlists change every few seconds, so they get a short time, but not zero. The origin sends these times in the Cache-Control header of each file, and caches such as nginx follow it unless told otherwise.

With a 1-second cache time, each edge can absorb thousands of playlist reloads a second and pass on one or two. Behind three regional shields, the origin should see roughly 15 playlist requests a second for five qualities. Last season it saw lakhs.

Request collapsing for new segments

When a new segment appears, thousands of requests for it reach an edge in the same instant, and the edge has no copy yet. Request collapsing lets only the first request go upstream. The rest wait for that one copy.

Follow the No arrow to see where the collapsing happens.

Here is the idea in Python, for one cache server. fetch_from_origin stands for the network call upstream. Each segment name gets an Event, a flag that other requests, each on its own thread, can wait on.

import threading

cache, inflight, lock = {}, {}, threading.Lock()

def get_segment(name):
    with lock:
        if name in cache:
            return cache[name]
        waiter = inflight.get(name)
        if waiter is None:                     # first asker becomes the leader
            inflight[name] = threading.Event()
    if waiter is not None:
        waiter.wait(timeout=2)                 # others wait for the leader
        if name in cache:
            return cache[name]
        raise TimeoutError(name)               # leader failed: the player retries later
    try:
        cache[name] = fetch_from_origin(name)  # the only trip upstream
    finally:
        with lock:
            inflight.pop(name).set()           # wake everyone waiting
    return cache[name]

The lock makes sure only one request becomes the leader for each segment. The finally block wakes the waiters even if the fetch fails. A waiter never goes upstream itself: if the leader failed or took over 2 seconds, it returns an error and the player retries. Set that wait above your normal fetch time.

That last rule came from a test. An earlier version let each waiter fetch for itself when the leader failed. With 500 threads and a failing origin, all 500 went upstream at once, exactly when the origin was struggling. With the rule, it was 1.

In nginx, a web server that can also act as a cache, this setting is called proxy_cache_lock. Its documentation says only one request at a time may fill a new cache entry, and others wait. After proxy_cache_lock_timeout, 5 seconds by default, a waiter goes upstream itself and its response is not cached.

Serving slightly old copies while a playlist refreshes

RFC 5861 defines stale-while-revalidate, which lets a cache serve an expired copy while it fetches a fresh one in the background. Its sister, stale-if-error, lets a cache serve the old copy when the origin returns a 500, 502, 503 or 504 error.

Pause and predict. A media playlist cached for 1 second has just expired, and 300 phones ask for it in the same instant. proxy_cache_lock is on. How many of those 300 requests reach the origin?

We ran it on nginx 1.24.0, the Ubuntu package, with a small Python origin that takes half a second to build the playlist. Every test ran three times with the same result each time.

What just happened? proxy_cache_lock covers only a new cache entry. When a cached playlist expires, every request that arrives before the fresh copy is stored goes upstream. The nginx docs split the job: the lock for new entries, proxy_cache_use_stale with its updating parameter for refreshes.

Add proxy_cache_background_update and nginx serves the old copy to everyone while one background request fetches the new one. Here is the edge setting the team now uses:

# inside the http block
proxy_cache_path /var/cache/nginx/edge levels=1:2 keys_zone=edge:10m
                 max_size=10g inactive=10m use_temp_path=off;

# inside the server block
location /live/ {
    proxy_pass http://origin;
    proxy_cache edge;
    proxy_cache_lock on;                # one fill for each new file
    proxy_cache_use_stale updating error timeout http_500 http_502 http_503 http_504;
    proxy_cache_background_update on;   # refresh expired copies in the background
    add_header X-Cache-Status $upstream_cache_status;
}

To check it on your own edge, run curl -sI against a playlist URL and look at the X-Cache-Status line. Under steady reloads you should mostly see HIT, with an occasional UPDATING or STALE as it refreshes. MISS on every request means the playlist is never cached.

That header exists only because of the add_header line above. Commercial CDNs report hits under their own header names, so check your CDN's documentation for the equivalent.

One more observation from the runs: with a 1-second cache time and stale serving on, nginx refreshed the playlist about every 2 seconds, not every second. The nginx 1.24.0 source explains why, with or without stale serving. It counts cache times in whole seconds and keeps an entry fresh until its last second has fully passed. So a viewer can get a playlist about 2 seconds old, which 6-second segments absorb. The docs do not mention this timing (as of 21 October 2026); pin your version.

You pre-scale: raise capacity to the expected peak plus a buffer well before the toss. Reactive autoscaling, which adds servers when load rises, acts only after the load arrives, and new servers can take minutes to boot and warm up. Predictive scaling forecasts from daily or weekly traffic patterns, and a one-off final does not follow one.

This is the water station stocked before the gun. Once the toss rush starts, a server that is still booting is no help.

Cloud providers support this directly. Amazon EC2 Auto Scaling, for example, offers scheduled actions that set a group's minimum, maximum and desired number of servers at a chosen time. Its documentation says an action generally runs within seconds but might be delayed by up to two minutes.

Scheduled scaling and normal autoscaling work together. The schedule sets a high floor before the match, and autoscaling can still add more if the crowd beats the forecast. Here is the team's plan for the final.

1. 1

   One week before: load test

   Replay a recorded toss rush at 1.5 times the forecast against a copy of the join path. Fix whatever breaks first, then test again.

2. 2

   Two days before: tell the CDNs

   The app uses two CDN companies. Both get the expected peak bandwidth and the match times, so they can place enough capacity in each region.

3. 3

   90 minutes before: scale up

   Scheduled actions raise API, login and entitlement servers to the forecast peak plus 30%. Autoscaling stays on above that floor.

4. 4

   30 minutes before: warm the caches

   A script requests the multivariant playlists and home screen data from every region, so the first real viewers hit warm caches.

5. 5

   During the match: freeze changes

   No deploys and no config changes, except the overload levers that the team has already practised.

Two CDNs also protect against one of them failing. The app gets a list of CDN addresses at startup. If one CDN returns errors, the player switches to the other for its next segment request.

It should protect people already watching first, let new viewers in at a rate the system can handle, and drop optional extras before core video. Each step should be a switch the team has practised.

Start at the dashboard and follow the questions.

Each switch has a reason behind it:

• Signed match token. After one successful check, the entitlement service gives the app a signed token valid until the match ends. Playback checks the signature without calling the service again. The cost: a refund mid-match takes effect only when the token expires.
• Retries with backoff and jitter. Backoff means waiting longer after each failure, and jitter adds a random extra wait. Without the randomness, lakhs of phones retry in the same second.
• Waiting room. New viewers see a short queue screen and are let in at a rate the join path can handle. People already watching are never sent back to it.
• Drop extras first. Live chat, polls and personalised rows on the home screen switch off before anything touches video.

Live score updates, the ball-by-ball text beside the video, are a separate problem with their own fan-out design. The team keeps them on separate servers, so a score surge never competes with video for capacity.

Explain it as a pipeline plus a peak plan. Draw the video path, show why caches carry it, then spend most of your time on the join rush and what you switch off under load.

1. 1

   Ask about the event

   Is it live or on demand? How many peak viewers, and how fast do they arrive? How far behind live is acceptable?

2. 2

   Do the rough maths

   Peak viewers times average bitrate gives CDN bandwidth. Peak joins per second times calls per join gives API load.

3. 3

   Draw the video path

   Encoder, transcoder with a ladder, packager, origin, origin shield, CDN edges, player. Name the segment length and why you chose it.

4. 4

   Protect the origin

   Short cache times for playlists, long ones for segments, request collapsing for new files and stale serving while expired playlists refresh.

5. 5

   Design the join path

   Signed tokens, caching what is not personal, pre-scaling before the known peak and a waiting room for the unknown one.

6. 6

   Plan the bad night

   Cap the top rung, add backoff and jitter to retries, drop extras and keep a second CDN ready.