Presence System Design: Heartbeats, Last Seen, Fan-Out and Privacy

Contacts see a green dot for someone whose phone died an hour ago, and your database groans under status writes. Both come from the most common mistake in presence system design: treating presence like normal data.

Presence is the small piece of state that says whether a person is reachable now, and when they last were. It changes every few seconds for millions of people, and most of it is worthless a minute later.

Picture a common setup. A chat app in India has 5 crore (50 million) daily users, and about 1 crore (10 million) phones are connected at the evening peak. The app and its numbers are made up, but every failure below is real.

The first version stores an is_online column in the users table. It sets the column to true when a phone connects and false when it disconnects. It also updates a last_seen column on every small ping the phone sends to keep its connection alive.

Three things break within a week:

• Phones that lose signal in a lift or run out of battery never send a goodbye, so the column stays true and contacts see them online for hours.
• Updating last_seen on every ping from 1 crore phones means hundreds of thousands of database writes per second, for a value nobody reads most of the time.
• Each status change is pushed to all of a user's contacts, around 300 on average, including people who are offline or never open that chat.

This first try is the naive version. The rest of this guide replaces each part of it, one problem at a time.

You need six words. Each one gets a plain meaning here and keeps the same meaning for the rest of the guide.

• Connection gateway: the server that holds a phone's open connection, usually a WebSocket, which is a two-way channel that stays open between the phone and the server.
• Heartbeat: a tiny message the phone sends on a fixed interval only to prove it is still there.
• TTL (time to live): a timer attached to a stored key. When the timer runs out, the store deletes the key by itself.
• Redis: a fast in-memory key-value store that supports TTLs. In-memory means it serves data from RAM rather than from disk, so reads and writes are very fast.
• Fan-out: copying one event to many receivers, like one status change going to every contact who should see it.
• Pub/sub (publish and subscribe): a messaging pattern where senders publish to a named channel and every current subscriber of that channel receives a copy.

Presence system design is the part of a chat app that answers four questions. Is this person online? When were they last seen? Who should hear about a change, and who is allowed to know?

Each question has its own failure. Detection fails when silent deaths look online. Storage fails when last seen writes swamp the database. Fan-out fails when one change floods thousands of phones. Privacy fails when the server leaks status to the wrong person.

The diagram shows the parts our chat app ends up with. Follow the heartbeat from the phone into the presence store, then notice that nothing tells contacts directly. Every change passes through the presence service first, because that is where privacy rules are applied.

The phone sends a heartbeat every 30 seconds, and the gateway answers each one by rewriting a Redis key with a 100 second TTL. While beats keep coming, the key never expires. When they stop for 100 seconds, the key deletes itself and the user reads as offline.

Think of a parking meter. Each heartbeat is a coin that resets the meter to full. The parking spot is the presence key, and an empty meter means the car has gone.

The picture also explains a surprise later in this guide. Anyone who looks at the meter sees at once that it has run out. But the warden who writes the ticket only notices when they walk past, and that walk can be slow.

Where the picture breaks: a real meter adds time for each coin. A presence key does not add up; every heartbeat resets it to the same full 100 seconds.

Why the server cannot wait for a disconnect

A phone that loses power or signal sends no goodbye. The WebSocket protocol, RFC 6455, defines a Close frame for a clean shutdown, but a dead phone cannot send it. The server may keep a socket that looks open long after the phone is gone.

The operating system's own check is far too slow for presence. On the Linux machine we tested (kernel 6.18), tcp_keepalive_time was 7200 seconds. That means two hours of silence before the first keepalive probe, a test packet the operating system sends on an idle connection. It only happens on sockets that switched keepalive on.

RFC 6455 also defines Ping and Pong frames. An endpoint that receives a Ping must answer with a Pong, and a Pong may be sent unprompted as a one-way heartbeat. The RFC sets no interval and no timeout, so the app must choose both.

What the gateway writes

Every heartbeat runs this Redis command: SET presence:USER GATEWAY EX 100. The value names the gateway holding the connection, so other servers know where to deliver messages. EX 100 attaches a fresh 100 second TTL each time.

You can watch this yourself with redis-cli, the Redis command-line client. Run SET presence:u42 gw-7 EX 100, then TTL presence:u42. You should see 100, counting down by one each second.

Two other replies matter. TTL returns -2 when the key does not exist, which means offline. It returns -1 when the key exists with no timer at all, which is a bug in a presence system.

The key loses its timer and the user stays online forever. The Redis EXPIRE documentation says a timeout is cleared by commands that overwrite the key, including SET. In our run on Redis 7.0.15, TTL went from 30 to -1 after a plain SET.

The opposite trap also exists. If you store presence in a Redis hash, HSET leaves the timer untouched, so it does not refresh it either. In our run, TTL kept counting down after HSET and only reset after an explicit EXPIRE. Refresh the timer on every beat, on purpose.

Choosing the heartbeat interval and TTL

The TTL must be a few heartbeats long, so one or two lost beats do not flip anyone offline. It must also be short enough that a dead phone disappears quickly. These two goals pull against each other.

With a heartbeat interval H and a TTL T, a phone that dies silently disappears between T minus H and T seconds later. The last beat landed at most one interval before the death, and the key lives for T after that beat.

We checked this on real Redis 7.0.15 with the timings scaled down to a 1 second heartbeat and a 3 second TTL. Across 200 phones that died at random moments, keys vanished between 2.02 and 3.00 seconds after death, exactly the predicted window.

To choose the ratio, we simulated heartbeats with 2% of them lost and up to 3 seconds of network delay. These inputs are made up, but the shape of the result is the point.

A TTL of exactly three beats still flips users when two beats in a row are lost. The next beat then only needs to land slightly later than the last good one. Adding 10 seconds of slack cut false flips about 25 times in the simulation. That is why our app uses 30 seconds and 100 seconds.

Here is the core in Python with redis-py, the standard Redis library for Python. The heartbeat writes two things: the online key with its TTL, and the beat time in a sorted set. A sorted set is a Redis collection kept in order by a number called a score. The sorted set feeds the sweeper in the next section.

import time
import redis

r = redis.Redis(decode_responses=True)
HEARTBEAT, TTL = 30, 100   # seconds

def heartbeat(user_id, gateway):
    pipe = r.pipeline()
    pipe.set(f"presence:{user_id}", gateway, ex=TTL)   # online flag
    pipe.zadd("presence:beats", {user_id: time.time()})  # last beat
    pipe.execute()

def is_online(user_id):
    return r.exists(f"presence:{user_id}") == 1

# Take every user whose last beat is older than TTL, atomically.
SWEEP = r.register_script("""
local gone = redis.call('ZRANGEBYSCORE', KEYS[1], '-inf', ARGV[1],
                        'WITHSCORES', 'LIMIT', 0, ARGV[2])
for i = 1, #gone, 2 do redis.call('ZREM', KEYS[1], gone[i]) end
return gone
""")

def sweep(batch=500):
    gone = SWEEP(keys=["presence:beats"], args=[time.time() - TTL, batch])
    # pairs of user id and last beat time = their last seen
    return [(gone[i], float(gone[i + 1])) for i in range(0, len(gone), 2)]

What just happened? heartbeat() refreshes both records in one round trip. is_online() is a single key check. sweep() pulls every user whose last beat is older than the TTL and removes them. It does this inside one Lua script, a small program that Redis runs as one step. If it returns a full batch, call it again until it returns fewer.

The Redis scripting documentation says it guarantees the script's atomic execution and blocks all other server activity while it runs. So if two sweeper copies run at once, each stale user is returned to only one of them. We ran this code on Redis 7.0.15 with redis-py 5.2.1.

The beat times come from each gateway's own clock, so keep gateway clocks synced. A few seconds of clock skew shifts when the sweeper fires by the same amount.

To check it, beat for one user, stop, and wait a little longer than the TTL. sweep() should return that user with a last beat time about TTL seconds old. A second sweep() should return an empty list.

No, not reliably. Redis can publish an event when a key expires, but its own documentation says the event comes when Redis deletes the key, not when the timer hits zero. On an instance holding many other keys with a TTL, we measured gaps of several minutes.

The feature is called keyspace notifications. It is off by default and is switched on with the notify-keyspace-events setting. A listener then subscribes to a channel such as __keyevent@0__:expired and receives the name of each expired key.

How Redis finds expired keys

The documentation names two ways. A key is found expired when a command touches it, or a background job finds it while scanning, a little at a time. A presence key nobody reads waits for the background job.

The redis.conf shipped with 7.0.15 says the default background effort tries to avoid more than ten percent of expired keys still sitting in memory. That target is about memory, not about how quickly each event goes out.

What we measured

We set 1,000 presence keys to expire within one second and timed each expired event against its deadline. We ran it on an idle Redis 7.0.15 with default settings, then added other keys with a one hour TTL, like sessions sharing the instance.

This is the parking meter again. Anyone who reads the key sees the expiry at once, but the warden, the background job that sends the event, walks past late.

Reads were never late. In the same 100,000-key setup, a GET one moment after the deadline returned nothing, and the expired event fired right after that read. The online check is accurate; only the push is late.

These delays are our own measurement on one machine; Redis documents only that the delay can be significant. Treat the exact numbers as undocumented as of 25 October 2026, and pin your version if you rely on them.

What to use instead

Announce offline from two places you control. The gateway announces a clean close itself, after a short grace period. The sweeper from the code above runs every few seconds and catches silent deaths within one sweep of the TTL running out.

The diagram shows the full path to "offline". Notice that both paths meet at the same question, because a phone that comes back in time should cause no announcement at all.

Is the grace period worth the short delay? We simulated 100,000 disconnects with made-up gaps: most phones back within 5 seconds, some within 30 seconds, the rest gone for minutes or hours. Every disconnect without a grace period caused two announcements, offline and then online.

• No grace period: 200,000 announcements.
• 10 second grace period: about 72,000 announcements, a 64% cut.
• 30 second grace period: about 40,000 announcements, but every real exit shows 30 seconds late.

Write last seen once, when a session really ends, and keep everything finer than that in memory. Last seen is a single timestamp per user, so it needs one durable write per session, not one per heartbeat.

Run the numbers for our app. Writing last_seen on every 30 second heartbeat for 1 crore connected phones is 3.3 lakh (333,000) writes per second at peak. Averaged over a day, with a made-up 3 hours online per user, it is still about 2.1 lakh (208,000) a second.

Writing once per session end, at a made-up 20 sessions per user per day across 5 crore users, averages about 11,600 writes per second. That is about 18 times fewer, comparing daily averages on both sides.

The sorted set already holds each user's latest beat time. When the sweeper or the gateway decides a user is offline, that last beat time becomes their last seen, and only then is it saved to the durable store.

• While a user is online, last seen is not shown, so there is nothing to store yet.
• On a clean close, save the close time after the grace period ends.
• On a silent death, save the last heartbeat time, not the moment the sweeper noticed. Otherwise last seen is wrong by up to 100 seconds.
• Batch the durable writes, for example one bulk write every second, so the database sees a steady trickle instead of a spike after a regional outage.

That last point matters most. If one mobile network drops in one state, lakhs of users go silent at once, and their TTLs run out within the same 30 second window. Batching turns that cliff into a queue your database can drain.

Send each change only to people who are watching that user right now, and send it through the gateways in batches. Pushing every change to every contact multiplies one event by hundreds, most of them wasted.

Who should hear about a change

Take a user with 300 contacts, a made-up but ordinary number. About 60 might be online, and maybe 5 have that user's chat or profile open. At 2,000 status changes a second across the app, the choice of audience decides the load.

The third row is a subscription model. When a phone opens a chat or a contact list screen, it subscribes to the presence of the people on that screen. When it leaves the screen, it drops those subscriptions.

Slack moved its real-time messaging API to this model in 2017. Its developer changelog announced that presence subscriptions would become mandatory on 15 November 2017. It explained that sending presence events for every user in a workspace is expensive. Each subscription request carries the full list of user IDs, and subscriptions end when the WebSocket disconnects.

Slack's engineering blog post on its edge cache, Flannel, reported that moving presence to pub/sub cut the presence events clients received by a factor of five. As of 25 October 2026, Slack's docs suggest 500 users as a good maximum per subscription list.

How the update reaches the right phones

The presence service does not talk to phones directly. It publishes to one pub/sub channel per gateway server, and each gateway forwards the update to the subscribed phones it holds. One change becomes one message per gateway, not one per phone.

Discord's 2017 engineering post describes the same pressure. When a user came online, their guild (what users call a Discord server) published that presence to every other connected session. Its fix, a library called Manifold, groups receivers by machine, so the sender makes at most one send per remote node.

Because pub/sub can drop messages, a phone should also fetch current presence when it opens a screen or reconnects. The pushed updates then keep that snapshot fresh, and a missed update fixes itself on the next screen open.

Privacy rules decide, on the server, who may see each user's online status and last seen. The check must run before every read and every fan-out, because a hidden status that still reaches the phone is not hidden.

WhatsApp's help centre is a good real reference, as of 25 October 2026. Users choose who sees their last seen and online: Everyone, My contacts, My contacts except chosen people, or Nobody. If you do not share yours, you cannot see other people's.

Three rules keep this safe. Filter on the server, never in the app. Run the same check in the read path and the fan-out path, so a hidden user cannot be discovered by polling. And apply reciprocity, the rule that hiding yours hides others' from you, in the same place.

The open chat standard XMPP, defined in RFC 6121, builds privacy into the protocol. Your server broadcasts your presence only to contacts whose subscription you approved. On login, it sends your presence to them and sends presence probes, requests for a contact's current status, on your behalf.

Mind the timing side channel too. If a hidden user's typing indicator or delivery ticks still update instantly, a determined viewer can guess they are online. Decide which signals the setting covers, and test each one.

Split presence by user ID across Redis nodes, keep one key per device, and let each gateway own the connections it holds. No single machine needs a full view of who is online.

Multi-device makes "online" a rule rather than a flag. A user with a phone and a laptop should read online if either one is connected. Slack's docs use this rule: a user is active if at least one client is connected and they are not marked away.

A simple way is one key per device, each with its own TTL. The user is online if any of their device keys exists. Keep the user's device list somewhere stable, then read all those keys with one MGET.

In Redis Cluster, the mode where keys are spread across many nodes, that MGET fails unless every key sits in the same slot. Put the user ID in a hash tag, curly braces inside the key name, such as presence:{USER}:phone.

We checked this on a local three-node Redis 7.0.15 cluster. MGET on presence:u42:phone and presence:u42:laptop returned a CROSSSLOT error. With the user ID in braces, both keys mapped to slot 11448 and MGET worked.

Plan for pub/sub in Redis Cluster too. Regular pub/sub sends every message to every node. The pub/sub docs say sharded pub/sub, added in Redis 7.0, keeps messages within one shard, which scales better for per-gateway channels.

Idle is a separate signal from online. Slack marks a user away after 10 minutes with no activity, according to its docs as of 25 October 2026. Track idle from app activity, not heartbeats, because a phone in a pocket can stay connected and beating.

Lead with the naive version, name what breaks, then walk through detection, storage, fan-out and privacy in that order. Give numbers at each step.

1. 1

   State the scale

   Say how many users are connected at peak and how many contacts each has. For example, 1 crore connected, 300 contacts each, heartbeat every 30 seconds.

2. 2

   Show the naive design and its failures

   An is_online column misses silent deaths, last_seen on every beat floods the database, and pushing to all contacts wastes most messages.

3. 3

   Explain heartbeats and TTL keys

   Each heartbeat rewrites a Redis key with a TTL a bit over three beats long. A silent death shows as offline within TTL seconds.

4. 4

   Explain how offline is announced

   The gateway announces clean closes after a grace period, and a sweeper catches silent deaths. Say why Redis expired events alone arrive too late.

5. 5

   Store last seen once per session

   Write the last heartbeat time to durable storage when the session ends, batched. Compare the two write rates out loud.

6. 6

   Fan out to watchers through gateways

   Phones subscribe to the people on screen. The service publishes one message per gateway, after the privacy check.

7. 7

   Close with privacy and failure modes

   Server-side filtering with reciprocity, a fresh snapshot on reconnect, and batching during a regional outage.