GitHub's October 2026 Actions Outage Explained: A Retry Storm on a Shared Link

When many servers lose a connection together and all reconnect at once, the reconnect traffic itself can become the outage. Engineers call this a reconnect storm, a kind of retry storm: many clients waking up and rushing one resource together.

GitHub's write-up of its 19 October incident (the GitHub Actions outage, October 2026) describes exactly this. Jobs were slow to start, logs and checks lagged, and GitHub Pages builds and webhooks were delayed. The trigger was small: lost connections to some cache servers in one cloud region.

This guide walks through GitHub's own account, line by line. Then it explains three ideas the outage teaches: a reconnect storm, a shared blast radius, and a safety limit left in shadow mode. Every incident number comes from GitHub's status page; durations and IST times are worked out from its UTC times.

Six terms carry the whole story. Each one is short to explain.

• Connection: an open line between two programs over the network. Opening one takes a handshake, a few messages back and forth, so opening many at once costs real network traffic.
• Cache server: a fast store that keeps data close to the code that needs it. Here, an Actions service that stores logs and run status talked to cache servers in a cloud region.
• Shared network link: one network path that several services use together. GitHub's link ran between its data centre (a building full of servers that GitHub runs) and a cloud region (a cloud provider's group of data centres).
• Retry and backoff: a retry tries a failed action again. Backoff makes each retry wait longer than the last. Jitter adds a random part to that wait, so clients do not stay in step.
• Rate limit: a rule that caps how many actions are allowed per second. GitHub's limit capped how quickly this service could open new connections.
• Shadow mode: a limit that checks every action and logs what it would have blocked, but blocks nothing. Teams use it to watch a new rule safely before switching it on.

For 88 minutes on 19 October 2026, from 15:24 to 16:52 UTC, GitHub Actions, GitHub Pages and webhooks generated by Actions were degraded. In India that was 20:54 to 22:22 IST, a Monday evening.

GitHub lists the impact in its resolved summary. These are its numbers, copied exactly.

• 5.13% of jobs on GitHub-hosted runners were delayed in starting. A runner is the machine that runs your workflow's job.
• 0.02% of workflow runs failed.
• During the first 30 minutes, 8.47% of requests to the Actions jobs REST API returned errors.
• Workflow status updates, logs and checks were delayed. Pages deployments and Actions-generated webhooks were delayed too.
• Live updates on GitHub.com and code search were also degraded for part of this time.
• GitHub states that no data was lost.

The timeline GitHub posted

Notice the gap between 15:24 and 15:49. GitHub's first status update came 25 minutes after the impact began, by GitHub's own start time. Keep that gap in mind when you judge an outage only by its status page.

GitHub says an internal Actions service lost connections to some of its cache servers, then retried them from every server at once. The service stores logs and run status, and the cache servers were in one cloud region. Each step below keeps to GitHub's own words.

The resulting surge of new connections overloaded a shared network link between GitHub's data centre and that region. That overload also disrupted other services that use the same link.

What GitHub has not explained yet

GitHub says it is continuing to investigate the cause of the initial connection timeouts. So the first step in the chain is still open, as of 26 October 2026. This guide does not guess at it, and you should not either when you retell the story.

The rest of the chain is explained, and it is the part worth learning. A small fault became a wide outage because of how the service reacted to it.

How GitHub stopped it

GitHub says it mitigated the incident in two moves. It moved the Actions service off the affected cache servers, and it reduced connections from other services on the link. It then processed the delayed Actions work.

Because they share the road. A network link has a fixed capacity, and every service using it gets slower when one service floods it. Engineers call the set of things one failure can damage its blast radius.

GitHub's summary says the surge disrupted other services that use the same link. It does not list which services shared the link, so this guide does not map each symptom to it.

An everyday picture: the stadium exit

Think of a cricket stadium where the floodlights fail and everyone leaves at once by one road. The fans are the servers, and the road is the shared link. The buses and ambulances on that road are the other services.

A staggered exit, block by block, is backoff with jitter. A marshal at the gate letting a few hundred people out per minute is an enforced rate limit. A marshal who only counts people and waves everyone through is shadow mode.

Where the picture breaks: fans on a jammed road still arrive in the end. A connection attempt that waits too long times out and is retried, so a jam on the network creates more traffic, not just slower traffic.

Shadow mode means a rule runs and records its decisions without acting on them. GitHub says a limit on how quickly this service can open new connections was being rolled out. During the incident it was still in shadow mode, logging only and not enforcing.

Teams use shadow mode for a good reason. A new limit set too low can block healthy traffic and cause its own outage. Watching the logs first shows what the limit would block on a normal day.

The cost is the gap before enforcement. During that gap the limit protects nothing, however well it is tuned. In GitHub's case the guard for exactly this failure was being rolled out, but only watching. The table below sets the two modes side by side.

GitHub has done two things: it enforced the new-connection limit and capped the service's concurrent connection attempts. Four more follow-ups are written as work in progress, as of 26 October 2026. The table lists all six from the resolved summary.

Read the list as layers. Backoff makes each server polite. The rate limit makes the whole service polite even if some code forgets. The capacity cap protects neighbours even if both of those fail.

Check the GitHub status page first, but expect it to lag. On 19 October its first post came 25 minutes after the impact began. Then avoid adding load. Most of the impact was delay, not failure: GitHub says 0.02% of workflow runs failed.

• Do not bulk re-run delayed workflows. Delayed jobs were processed after mitigation, and mass re-runs add to the queue.
• Re-run only runs that actually failed, after the status page shows recovery.
• If your scripts poll the Actions REST API, back off on errors. GitHub says 8.47% of requests to the Actions jobs REST API returned errors in the first 30 minutes.
• If your service consumes webhooks, expect late events. Handle each event safely even if it arrives late or twice.
• For deploys that cannot wait, keep a documented manual path that does not depend on CI.

Run a small simulation. The script below is a toy model with made-up numbers, not GitHub's system. It shows the shape of the problem, so you can see why backoff and an enforced limit change the outcome.

The setup is made up: 500 servers each lose 4 connections at once, and the cache stays unreachable for the first 2 seconds. Every reconnect crosses one link that carries 400 exchanges per 100 ms tick. Another service sends 100 requests per tick over it.

When more is offered than the link carries, packets are dropped for everyone. A handshake needs 3 packets through, so handshakes suffer more than single requests. The limit allows 250 new connections per tick across the service. Case 6 adds a random wait of up to 2 seconds before each connection's first reconnect.

"""Reconnect storm simulation. Every number here is made up; it only shows the shape of the problem.

A service runs on many servers. Each server loses its cache connections at tick 0 and must reopen them.
All new connections cross one shared network link, which another service also uses.
When more traffic is offered than the link can carry, packets are dropped for everyone.
A connection handshake needs 3 packets to get through, so it is hit harder than one packet.
The cache is unreachable for the first 2 seconds, so every attempt in that time fails.
"""
import random

SERVERS = 500           # made up
CONNS_PER_SERVER = 4    # made up
LINK_CAPACITY = 400     # exchanges the link carries per tick (made up); 1 tick = 100 ms (made up)
OTHER_SERVICE = 100     # other service's requests per tick on the same link (made up)
PACKETS_PER_HANDSHAKE = 3
RATE_LIMIT = 250        # service-wide new connections allowed per tick (made up)
TICKS = 300             # 30 seconds
CACHE_DOWN_TICKS = 20   # cache unreachable for the first 2 s, so every attempt fails then (made up)
WINDOW = 100            # measure the other service over the first 10 s


def run(backoff, limit_mode, first_jitter=False, seed=7):
    rng = random.Random(seed)
    n = SERVERS * CONNS_PER_SERVER
    if first_jitter:
        pending = [rng.randrange(20) for _ in range(n)]  # random wait of 0-2 s before the first reconnect
    else:
        pending = [0] * n  # tick of next attempt: everyone reconnects at tick 0
    attempt_no = [0] * len(pending)
    done = 0
    peak = total = blocked = would_block = 0
    other_ok = other_sent = 0
    worst_other = 1.0
    finished_at = None
    for t in range(TICKS):
        due = [i for i, nxt in enumerate(pending) if nxt is not None and nxt <= t]
        rng.shuffle(due)
        if len(due) > RATE_LIMIT:
            if limit_mode == "shadow":
                would_block += len(due) - RATE_LIMIT  # logged only
            elif limit_mode == "enforced":
                held = due[RATE_LIMIT:]
                blocked += len(held)
                for i in held:
                    pending[i] = t + 1  # wait locally, nothing sent on the link
                due = due[:RATE_LIMIT]
        offered = len(due) + OTHER_SERVICE
        p = min(1.0, LINK_CAPACITY / offered)  # chance one packet gets through this tick
        peak = max(peak, len(due))
        total += len(due)
        other_success = sum(rng.random() < p for _ in range(OTHER_SERVICE))
        if t < WINDOW:
            other_ok += other_success
            other_sent += OTHER_SERVICE
        worst_other = min(worst_other, other_success / OTHER_SERVICE)
        for i in due:
            if t >= CACHE_DOWN_TICKS and rng.random() < p ** PACKETS_PER_HANDSHAKE:
                pending[i] = None
                done += 1
            else:
                attempt_no[i] += 1
                if backoff:
                    cap = min(64, 2 ** attempt_no[i])        # exponential, capped at 64 ticks
                    pending[i] = t + 1 + rng.randrange(cap)  # full jitter: random wait in [0, cap)
                else:
                    pending[i] = t + 1                       # retry on the very next tick
        if done == len(pending) and finished_at is None:
            finished_at = t
    return {
        "reconnected": f"{done}/{len(pending)}",
        "all back by": f"{(finished_at + 1) / 10:.1f} s" if finished_at is not None else f"not within {TICKS / 10:.0f} s",
        "peak attempts/tick": peak,
        "total attempts": total,
        "other service ok, 0-10 s": f"{100 * other_ok / other_sent:.1f}%",
        "other service worst tick": f"{100 * worst_other:.0f}%",
        "held by limiter": blocked,
        "shadow log: would block": would_block,
    }


if __name__ == "__main__":
    print("SIMULATION WITH MADE-UP NUMBERS: 500 servers x 4 connections, link 400/tick, other service 100/tick, cache down for first 2 s, 1 tick = 100 ms")
    cases = [
        ("1. Retry at once, no limit", False, "none", False),
        ("2. Retry at once, limit in shadow mode", False, "shadow", False),
        ("3. Retry at once, limit enforced", False, "enforced", False),
        ("4. Backoff after failures only, no limit", True, "none", False),
        ("5. Backoff after failures + limit enforced", True, "enforced", False),
        ("6. Jitter before first reconnect + backoff, no limit", True, "none", True),
    ]
    for name, b, mode, first in cases:
        r = run(b, mode, first)
        print(name)
        for k, v in r.items():
            print(f"    {k:27} {v}")

1. 1

   Save the file

   Save the code as reconnect_storm_sim.py. It uses only the Python standard library.

2. 2

   Run it

   Run python3 reconnect_storm_sim.py from the same folder. It finishes in about a second on a laptop.

3. 3

   Check the output

   We ran it on Python 3.11.17, 3.12.3 and 3.13.17 and got identical output, because the random seed is fixed. Compare your numbers with the table below.

What the numbers show

Case 1 is the natural first instinct, and the table shows why it is the wrong one. Cases 1 and 2 print identical numbers except the shadow-log line. In shadow mode the limit logged 96,979 attempts it would have blocked, and the other service still got only 47.2% of its requests through.

Backoff after failures, in case 4, cut total attempts from 114,909 to 11,526. The other service got 95.8% through in the first 10 seconds. But case 4 waits only after a failed attempt, so the first wave still hit the link at full size and its worst tick was 16%.

Case 6 puts jitter before the first reconnect too. The peak fell from 2000 to 475 attempts per tick and the worst tick rose to 66%. Jitter spreads the first wave out but does not cap it.

Only the enforced limit kept every tick safe, because 250 plus the other service's 100 stays under the link's 400. In case 3 it held attempts back 42,000 times; a held attempt waits on the server and never touches the link. It also brought everyone back faster than no limit, 2.8 s against 7.2 s, because the link never collapsed.

What the simulation does not show

• It is not a model of GitHub. GitHub has not published server counts, link capacity or retry settings, and none of these numbers come from it.
• The packet-loss rule is a simplification. Real networks queue, drop and retransmit in more complex ways.
• The limit here is one counter for the whole service. A real fleet often has to split a limit across servers, which is harder to tune.
• Backoff finished slower than retrying at once in this run. That trade is real: backoff gives up some speed to protect everyone else. In our reading, real systems still want backoff for two reasons. A limit keeps spending its full allowance on a dependency that is down: 5,000 attempts in 2 s in cases 3 and 5. In our check with a 10 s outage, backoff cut that from 25,000 to 14,229. And if a split limiter rejects instead of queueing, backoff stops each server retrying the rejection in a tight loop.

Treat reconnects as retries and give them the same care. It is easy to add backoff to request retries and forget the code that reopens connections. A dependency is any service your code needs, such as a cache or database.

1. 1

   Add jittered backoff to reconnects

   Wait a random time between zero and a growing cap before each reconnect, including the first one. Look in your client library for reconnect delay or backoff settings before writing your own.

2. 2

   Cap concurrent connection attempts

   Allow only a few handshakes in flight per process. This is the same kind of cap GitHub added.

3. 3

   Enforce a service-wide rate limit

   Pick a ceiling for new connections per second across the service, below what shared links can carry. Shadow-test it, then enforce it.

4. 4

   Budget shared capacity

   Know which links your service shares and how much of each it may use. Ask the network team for a per-service cap where possible.

5. 5

   Alert on connection-open rate

   Graph new connections per second per service. A sudden jump points at the noisy service while the surge is still happening.

Shadow mode in practice: Envoy and a rollout plan

The same idea appears in Envoy, an open-source proxy. In the Envoy v1.40.0 docs, its local rate limit filter has two separate fractions, set in config as filter_enabled and filter_enforced. The runtime settings describe the first as requests that check the decision but do not enforce it.

The second is the share that enforces it, which the docs say can be used to test what would happen before fully enforcing. Envoy also counts the two cases apart. The filter uses a token bucket, where each token is one unit of allowance.

Its rate_limited counter counts responses that found no free token, whether or not they were blocked. Its enforced counter counts requests it actually limited, for example with an HTTP 429 Too Many Requests reply. Moving a limit from shadow mode to enforced can follow five steps.

1. 1

   Ship the limit in shadow mode

   Log every action the limit would block, with the service name and a timestamp.

2. 2

   Compare logs with normal traffic

   Check a few normal days and any recent spikes. The limit should only trip on behaviour you would call a fault.

3. 3

   Set an end date

   Write down when the limit will be enforced and who decides. Shadow mode with no date can last until the next incident.

4. 4

   Enforce in stages

   Turn it on for a small share of servers first, watch errors, then widen it.

5. 5

   Alert on the shadow log meanwhile

   If the would-block count jumps, page someone. The log is useful only if a person sees it during the surge.

Use it as a short case study when asked about retries, rate limits or failure isolation. Keep to what GitHub stated and say what is still open.

1. 1

   State the facts

   On 19 October 2026, 15:24 to 16:52 UTC, GitHub Actions, Pages and webhooks were degraded. 5.13% of jobs on GitHub-hosted runners were delayed in starting.

2. 2

   Walk the chain

   Lost cache connections, retries from every server at once, a surge that overloaded a shared link, and other services hurt too.

3. 3

   Name the open question

   Say whether GitHub has explained the first connection timeouts; as of 26 October 2026 it had not. Interviewers notice when you separate facts from guesses.

4. 4

   Name three fixes in layers

   Jittered backoff per server, including before the first reconnect, an enforced service-wide connection limit, and a cap on each service's share of the link.

5. 5

   Discuss shadow mode

   Explain why teams shadow-test limits, and the risk of staying there.

6. 6

   Close with detection

   Graph connection-open rate per service so the source of a surge is found in minutes.