URL Shortener System Design: Keys, Redirects and Clicks

The obvious first version of a URL shortener breaks in three places at once, so good URL shortener system design starts there. Two links get the same short code, click counts come out too low, and redirects slow down at the busiest minute.

Picture a common setup. A link-shortening service in Noida sends SMS marketing for brands. Each sale message carries a short link instead of a long web address. The setup is made up, but the problems are real.

Short links matter a lot in SMS. One single SMS written in GSM-7, the basic character set phones use for Latin text, holds 160 characters. A single SMS with Hindi or other non-Latin text holds only 70.

A long address with tracking tags can eat most of that space. A short link such as short.example/xMnQGsz takes 21 characters and leaves room for the offer itself.

Every SMS gets its own personal link, so the brand can see which customers tapped. A campaign to 20 lakh (2 million) customers creates 20 lakh short links in one go.

The first version was built in two weeks. Then a shoe brand sent its Diwali sale at 7 pm, and three things went wrong.

• Some customers tapped the shoe offer and landed on a loan page from another campaign. Two long links had been given the same short code, and the newer one overwrote the older.
• The click report showed far fewer clicks than the brand's own website counted. Some phones were opening the shop page without asking the shortener at all.
• Redirects slowed to several seconds at the peak. Every tap also added one to the campaign's click total, on a single database row, and those writes queued up behind each other.

Each failure has a cause you can fix: how keys are made, which redirect is sent, and where clicks are counted. The rest of this guide fixes them in that order.

A URL shortener stores pairs: a short code and the long address it stands for. When someone opens the short link, the service looks up the code and tells the browser to go to the long address.

Think of an old landline phone with speed-dial. You press 2, and the phone dials a long number stored behind that button. The short code is the button, and the long URL is the number behind it.

Where the picture breaks: speed-dial lives inside your own phone. A short code lives on the shortener's servers, so every tap is a request over the internet. That request is what lets the service count clicks and block bad links.

Long URL. The full web address the brand wants customers to reach, often with tracking tags at the end. Example: a 180-character sale page address for the shoe brand.

Short code. The few characters at the end of a short link, such as xMnQGsz. It is the key the service looks up, so it must point to exactly one long URL.

Redirect. An HTTP response that tells the browser to go somewhere else. It carries a status code, such as 301 or 302, and a Location header (a labelled line in the response) holding the new address.

Click event. A small record written each time someone opens a short link: which code, when, and what kind of app or browser opened it.

Approach URL shortener system design as three separate paths: a create path that makes keys, a redirect path that must be fast, and an analytics path that counts clicks later. Keeping them apart fixes all three Diwali failures.

Start with the requirements, written as plain promises.

• Create short links in bulk for a campaign, and single custom links such as diwali-shoes for posters.
• Redirect every tap to the right long URL, quickly, even during a 7 pm blast.
• Never give two links the same code, and never reuse an old code.
• Let links expire after 90 days, and let the team block any link at once.
• Show each brand its clicks per campaign, per hour and per city, a few minutes behind real time.

A short link sits in 20 lakh inboxes and cannot be recalled. If redirects fail, every one of those links is broken, so the redirect path gets the most care.

Here is the whole system. Follow the customer's tap from the phone, and notice that the redirect service never talks to the analytics workers directly. The click queue sits between them.

It depends on who makes the links. This service creates about 1 crore (10 million) links a day but serves only about 50 lakh clicks, so it writes more than it reads.

The inputs below are made up, but the arithmetic is real. Divide by 86,400 seconds in a day to get rates.

Interview answers often assume people open short links far more often than they create them. That holds for a link shared on social media. It flips for SMS, where every customer gets a personal link and most are never tapped.

Give every new link a unique number from a counter, then write that number in base62, using digits, small letters and capital letters. Different numbers always give different codes, so generated codes cannot clash.

Base62 is counting with 62 symbols instead of 10. Each extra character multiplies the choices by 62. Seven characters give 62 to the power 7, which is 3,521,614,606,208 codes, about 3.5 trillion.

At 1 crore links a day, about 365 crore (3.65 billion) a year, seven characters last about 960 years. Six characters give about 5,680 crore (56.8 billion) codes, which would run out in about 15 years. So seven it is.

Why cutting a hash short goes wrong

The first version ran each long URL through MD5, a common hash function, which turns any text into a 32-character fingerprint in hex. It kept the first 6 hex characters (digits 0 to 9 and letters a to f). Six hex characters allow 16,777,216 codes, about 1.68 crore.

Pause and predict: with 1.68 crore possible codes, how many links can you make before a clash becomes more likely than not? A natural guess is somewhere in the lakhs.

The answer is about 4,800 links. This is the birthday problem: in a room of just 23 people, two probably share a birthday. Clashes start long before the space is full, roughly at the square root of its size.

You can check this yourself. The script below hashes 20 lakh made-up URLs the same way and counts how many land on a code already taken.

import hashlib

seen, clashes = set(), 0
for i in range(2_000_000):
    url = f"https://shop.example/sale?uid={i}"
    code = hashlib.md5(url.encode()).hexdigest()[:6]
    clashes += code in seen
    seen.add(code)
print(clashes)   # 114600

What just happened? It prints 114600, so about 1.15 lakh (115,000) of the 20 lakh links would collide. Probability theory predicts 114,610, so the run matches. In 200 separate runs, the median first clash came at link 4,744.

You can catch a clash with a conditional write, which fails if the code already exists, and retry. With 6 characters, though, retries start early. The table compares the common ways to make keys.

Handing out numbers without a bottleneck

Asking one central counter for every single link would make that counter a bottleneck. So each create server takes a block of 1,000 numbers at a time, using INCRBY in Redis, an in-memory data store.

INCRBY adds a number to a stored counter and returns the new value. Redis documents its increments as atomic, so two servers asking at the same moment get different blocks. The value is limited to 64-bit signed integers.

You can check it with redis-cli. Running INCRBY link_id 1000 twice on a fresh key returns 1000, then 2000, so the first server owns 1 to 1,000 and the second owns 1,001 to 2,000.

If a create server crashes, the rest of its block is simply never used. Gaps are fine, and duplicates are not. So the counter must survive restarts, and must never move backwards.

Here is the catch an expert will raise. Redis replicates to its standby copies asynchronously by default. Its documentation warns that acknowledged writes can still be lost during a failover, when a standby takes over from a failed main copy. That standby may hold an older counter.

That older counter would hand out numbers that were already used. So save every code with a conditional write, which fails if the code already exists. A repeated number then becomes a loud error, not a silent overwrite.

In Redis that is SET with the NX option, and other databases have their own form. Our test on Redis 7.0.15 showed the second SET NX for the same code returned nil and left the first URL in place.

Making codes hard to guess

Plain counting makes codes predictable, because one code is followed by the next. Anyone could walk through codes and open other customers' personal links.

OWASP, a non-profit that publishes security guidance, names this risk. Its redirect cheat sheet says mapping short IDs to URLs must not let a user cycle through IDs to find every target.

So the service scrambles each number before turning it into a code. It multiplies the number by a large secret and keeps the remainder after dividing by 62 to the power 7. The code below runs as it is.

ALPHABET = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
CODE_LENGTH = 7
KEY_SPACE = 62 ** CODE_LENGTH          # 3,521,614,606,208 codes
SECRET = 1_918_734_562_211             # odd, not a multiple of 31

def to_code(link_id):
    number = (link_id * SECRET) % KEY_SPACE   # scramble, still one-to-one
    chars = []
    for _ in range(CODE_LENGTH):
        number, rest = divmod(number, 62)
        chars.append(ALPHABET[rest])
    return "".join(reversed(chars))

print(to_code(1), to_code(2), to_code(3))  # xMnQGsz 5yLHmV8 Dl9y3nH

The secret line matters most. 62 is 2 times 31, so 62 to the power 7 is built only from the factors 2 and 31. A secret that is odd and not a multiple of 31 shares no factor with it.

Multiplying by such a secret never sends two IDs below 3.5 trillion to the same code. We checked every number in a smaller space of 62 to the power 4, and each one got its own code.

We also tried a secret of 217, which is 7 times 31, in that same smaller space. It gave only 476,656 different results for 14,776,336 inputs, so most codes would clash.

If guessable codes would leak something private, use random 7-character codes with a conditional write, and retry on the rare clash. You pay one extra failed write now and then, instead of a predictable pattern.

Random codes are not unguessable either. With 90 crore live links among 3.5 trillion codes, a random guess hits a live link about once in 3,900 tries. So keep the per-device limit on unknown codes.

Custom codes such as diwali-shoes are for posters and print ads. The service requires them to be at least 8 characters long, so they can never collide with generated 7-character codes. Saving one also uses a conditional write.

Use a 302 when you need click counts or may change or block the destination later. A 301 says the move is permanent, so browsers may store it and stop asking you.

MDN, the web documentation project, lists 301 among the status codes that are cacheable by default. A browser or proxy (a server between the phone and you) may store and reuse such a response without asking your server, unless headers say otherwise. 302 is not on that list.

A 302 says the page has moved for now. So the browser treats the short link as the real address and comes back to it next time.

Back to the landline. A 302 is like pressing 2 every time, so the phone always checks what button 2 holds. A 301 is like the phone quietly saving the long number after the first press. Change button 2 later, and that phone keeps dialling the old number.

That explains the missing clicks. The first version sent 301s, so repeat taps from the same phone could go straight to the shop. The fix was a 302 plus a Cache-Control: no-store header, which tells browsers not to store the response.

The switch did not rescue phones that had already stored a 301. MDN notes that no-store does not delete a response already stored for the same URL. Those phones kept skipping the shortener until their copy expired or was cleared.

• 301 Moved Permanently: cacheable by default. Right for a page that has really moved, wrong when you count clicks.
• 302 Found: temporary, so the browser comes back to you each time. The safer choice when you count clicks.
• 307 and 308: like 302 and 301, but the browser must keep the request method and body. That matters for form posts, not taps.
• MDN notes that search crawlers update the old address for permanent redirects, but do not memorise temporary ones.

To check your own service, run curl -sI followed by one of your short links. The status line should show 302, followed by a location header holding the long URL and cache-control: no-store.

Header names may print in lower case, as they do over HTTP/2, and HTTP/2 shows no word after 302. On our local test server the lines read 302 Found, the Location header and Cache-Control: no-store.

Make each redirect a single key lookup: check an in-memory cache first, fall back to a key-value database, and do nothing else before replying. Everything slow happens after the reply has gone.

Every read is by one code, so a key-value store fits well: a database that saves each value under a key and fetches it by that key. Amazon DynamoDB or Apache Cassandra would work, and so would a relational database split into shards (separate slices of the data) by code.

If you split shards by ranges of code, scrambled codes spread new links evenly. Plain counting would send every new link to the last range. If you split by a hash of the code, either kind spreads evenly.

Here is one tap, step by step. Only a cache miss reaches the database, and both paths end in the same reply.

There is a catch with personal SMS links. Each one is tapped only a few times, so a normal cache rarely sees the same code twice before it is evicted.

Most taps come within hours of the SMS landing. So at send time, the service loads the campaign's codes into the cache with a 48-hour expiry.

We measured this on Redis 7.0.15. Twenty lakh entries, each a 7-character code and a 173-character URL with a 48-hour expiry, used about 610 MB, roughly 300 bytes each.

To size your own entries, run MEMORY USAGE with one code in redis-cli. It returns bytes. Our single entry with a 192-character URL reported 280.

Here is the redirect handler, kept small. Plain dictionaries and a list stand in for the cache, the links database and the queue client, so the code runs as it is.

import time

cache = {}                 # code -> (long_url, cached_until)
links_db = {"xMnQGsz": {"long_url": "https://shop.example/sale",
                        "blocked": False,
                        "expires_at": time.time() + 90 * 86400}}
click_buffer = []          # stands in for the queue client's local buffer

def redirect(code, user_agent=""):
    now = time.time()
    hit = cache.get(code)
    if hit and hit[1] > now:
        long_url = hit[0]
    else:                                      # cache miss
        row = links_db.get(code)
        if row is None or row["blocked"] or row["expires_at"] <= now:
            return 404, {}
        long_url = row["long_url"]
        cache[code] = (long_url, min(now + 48 * 3600, row["expires_at"]))
    click_buffer.append({"code": code, "at": now, "agent": user_agent})
    return 302, {"Location": long_url, "Cache-Control": "no-store"}

print(redirect("xMnQGsz"))  # (302, {'Location': 'https://shop.example/sale', 'Cache-Control': 'no-store'})
print(redirect("nope123"))  # (404, {})

What just happened? Three lines matter. The min call keeps a cached entry from outliving the link's own expiry. The append only drops the click into a local buffer, so a slow queue never slows a tap.

The last line sends the 302 with no-store, so browsers ask again next time. In production, make sure a full buffer drops the event rather than blocking the reply.

Blocking has one more step. The block switch must delete the code from the cache too, or a cached copy keeps redirecting for up to 48 hours.

Record each click as a small event on a queue, and let separate workers count the events. The redirect never waits for a database write, and the reports can run a minute behind without hurting anyone.

The first version did the opposite. At 3,000 taps a second, each update to the campaign's single total row waited for the one before it.

Now each event goes to Kafka, an event log that works like a message queue. It stores events and lets workers read them at their own pace. Events go into a topic, a named stream, which Kafka splits into partitions. Kafka keeps events in the order they were written within each partition, not across them.

Kafka keeps events for a retention period you set per topic. The team keeps raw click events for 7 days, so a fixed worker can replay a bad day. In the diagram, see how the bot filter sits before any counting happens.

1. 1

   Filter out machines

   Not every request is a person. Link-preview robots, security scanners and crawlers also fetch links. The filter drops them by user agent (the User-Agent header, a labelled line in the request naming the app) and timing.

2. 2

   Count in memory first

   The minute counter adds up taps per code in memory. It writes one row per code per minute, instead of one write per tap.

3. 3

   Count unique people cheaply

   For personal links, unique customers equal the codes tapped at least once. Shared poster links need an estimate of unique devices instead.

User agents are easy to check. Slack's link-preview robot, for example, sends a user agent that starts with Slackbot-LinkExpanding. Bots that hide their name need timing rules, so treat bot filtering as an estimate.

Counting unique devices normally means remembering every device you have seen. Redis HyperLogLog estimates the count instead. Its documentation says it uses up to 12 KB with a standard error of 0.81%, and it never stores the items themselves.

Our test agreed. A million made-up devices came back as 996,852, which is 0.31% low, and the key held 12,304 bytes: 12 KB of counters plus a 16-byte header.

Privacy shapes this too. The Redis documentation notes that storing IP addresses or other personal identifiers is against the law in some countries.

So the filter feeds in a keyed hash of the IP address and user agent, using a secret key that changes every day. A plain hash of an IPv4 address is easy to reverse by trying all 4.3 billion addresses, so the key matters. A daily key also means unique counts are per day. Many phones on mobile data share one public IPv4 address, which undercounts, while one phone's address can change, which overcounts.

Limit who can create links, check every destination before saving it, and keep the power to block any link instantly. Short links hide where they go, which makes them useful for phishing (tricking people into giving away passwords).

The OWASP redirect cheat sheet explains the risk. A link on a trusted domain looks safe, so an attacker can use it to send people to a fake site and steal their credentials. A shortener is exactly that kind of trusted starting point.

• Only verified brand accounts can create links. There is no public form for anyone to use.
• Each brand registers its own domains. A destination's host must match one of them, compared after parsing the URL, never by checking how the text starts.
• New destinations are checked against known phishing and malware lists before the code is saved.
• Every link and campaign has a block switch, which also deletes the codes from the cache.
• One device may try only a few unknown codes a minute, which slows anyone walking through codes.
• Expired codes are never handed out again, because an old SMS can sit in an inbox for years.

OWASP also suggests a notice page that shows the destination and asks the user to confirm before leaving. The service offers one for links reported as suspicious.

Indian rules matter here too. In August 2024, TRAI, India's telecom regulator, directed operators to block business SMS carrying links that are not whitelisted. The rule took effect from 1 October 2024.

So the short-link domain itself must be on each brand's approved list with its SMS provider. Rules like this change, so confirm the current process with your provider before a launch.

Explain it as three paths, and back each choice with a number or a failure it prevents.

1. 1

   Ask who makes the links

   Shared links make the system read-heavy. Personal SMS links make it write-heavy. Say which one you are designing for.

2. 2

   Estimate out loud

   Links and clicks per day, storage per year, and the code length the key space needs.

3. 3

   Define two calls

   Create takes a long URL and returns a code. Redirect takes a code and returns a 302.

4. 4

   Choose how keys are made

   A counter handed out in blocks, scrambled and written in base62. Say why cutting a hash short clashes after a few thousand links.

5. 5

   Protect the counter

   Conditional writes turn a repeated number into an error, even if a failover moves the counter backwards.

6. 6

   Draw the read path

   Cache first, then a key-value store sharded by code, warmed when a campaign is sent.

7. 7

   Pick the status code

   Choose 302 for counting and blocking, and say what a 301 would cost.

8. 8

   Move analytics off the path

   Click events go on a queue. Workers filter bots, count per minute and estimate unique visitors.

9. 9

   Close with abuse and expiry

   Verified creators, destination checks, a block switch that clears the cache, and codes never reused.