Circuit Breaker Pattern: Fail Fast When a Dependency Is Sick

Every request that needs the slow dependency waits for it and holds a server worker while it waits. When every worker is waiting, nothing is left for anyone else, so healthy pages fail too. The circuit breaker pattern exists to stop exactly this.

Picture a common setup. An online home-decor marketplace sells lamps, cushions and brass diyas from small sellers. The setup is made up, but the problem is real.

Every card and UPI payment (India's instant bank-to-bank system) goes through a payment gateway, an outside company that moves money between the shopper's bank and the shop. The gateway is a dependency: checkout cannot finish without it.

On the Friday before Diwali, the sale went live at 8 pm. At 8:12, the gateway started having trouble. Calls that normally took 400 milliseconds now hung with no answer.

A timeout is the longest your code will wait for an answer before giving up. The shop's timeout for the gateway was 30 seconds. On top of that, each failed charge was retried twice.

The on-call engineer watched the dashboard turn red. First checkout failed, then product pages stopped loading, then search went down. By 8:20 the whole site showed errors, though only the gateway was sick.

The cause was arithmetic. A thread is one worker inside a server that handles one request at a time. The shop runs 6 servers with 50 threads each, so 300 threads in total.

• On a normal evening, about 4 checkouts arrive per second. Each holds a thread for 0.4 seconds, so about 1.6 threads are busy with payments at any moment.
• That night, each attempt waited the full 30 seconds and was tried 3 times. One checkout could hold a thread for 90 seconds.
• Four checkouts a second for 90 seconds needs 360 threads. The shop had 300, so every thread ended up waiting on the gateway.

The busy-thread numbers come from Little's law: average requests in progress equal the arrival rate times the time each one takes. You can redo the sums with your own traffic.

Here is what the servers looked like at 8:20. Look at where the threads are, and at the two healthy boxes nobody could reach.

The site was down for 40 minutes, costing about 18 lakh rupees (1.8 million rupees) in lost orders (made up). The review asked one question. Why did a payments problem stop people from even looking at lamps?

This is a cascading failure: one part fails, and the failure spreads to parts that were healthy. The rest of this guide builds the fix, a circuit breaker, and shows how to tune it.

The circuit breaker pattern is a wrapper around calls to a dependency that stops making those calls while they keep failing. Instead of waiting, your code gets an instant error and can do something useful.

Martin Fowler's 2014 article on the pattern credits Michael Nygard's book Release It! with making it popular. Its goal is to stop failures in remote calls from cascading through a system.

The name comes from the circuit breaker in a building's wiring, which cuts the current when something goes wrong. A sick dependency is one that is down, very slow or returning errors.

Think of a plumber who keeps not turning up. The first time, you take the morning off and wait at home. The second time, you wait again. After the third no-show, you stop calling him.

For a week you go straight to plan B. You fix small leaks yourself or call someone else. After the week, you try him once, on a small job like a dripping tap. If he comes, you go back to calling him.

Each part of that story maps to the pattern:

• Waiting all morning for him is a call stuck until its timeout. Your lost morning is the server thread.
• Not calling him for a week is the open state. You fail fast without waiting at all.
• The small trial job is the half-open state. It is one cheap test, not your big bathroom repair.
• Plan B is the fallback. Someone still has to decide what it is.

Where the picture breaks: a software breaker counts many calls, not three no-shows, and it opens for seconds, not a week. It also tests recovery by itself, while you have to remember to call the plumber.

A circuit breaker works as a small state machine with three states: closed, open and half-open. A state machine is something that is always in exactly one state and moves between states by fixed rules.

Closed: calls go through

In the closed state, every call reaches the gateway as normal. The breaker quietly counts successes and failures. When failures cross a limit you set, it trips to open.

Open: calls fail at once

In the open state, the breaker rejects every call immediately with an error, without touching the gateway. It stays open for a wait time, often called the break duration or reset timeout, such as 30 seconds.

Half-open: a few trial calls

When the wait ends, the breaker moves to half-open and lets trial calls through. If they succeed, it closes. If they fail, it opens again for another wait.

Libraries differ on how many trial calls they allow. Hystrix, Netflix's older Java library, lets the next single request through. Polly 8.8.0, a .NET library, also lets one probe through and rejects other calls while it runs.

Resilience4j, a current Java library, allows 10 trial calls by default and decides on their failure rate. Fowler's sample code does not limit trials at all: any call made while half-open goes through.

Follow the arrows from the checkout request below. Notice that the only way back to closed runs through a trial call.

Here is the same sale night again, replayed with a breaker on each server, a 3-second timeout and the same two retries. The times come from a small simulation of the scenario (made up).

1. 1

   8:12:00, closed

   The gateway starts hanging. Each failed attempt now holds a thread for 3 seconds, not 30.

2. 2

   8:12:08, trips open

   Each server has seen five failures in a row, between 7.5 and 9 seconds in. Its breaker opens, and new checkouts get an instant error and the fallback.

3. 3

   8:12:41, half-open

   Thirty seconds after the last failure, the next few checkouts go through as trial calls. They time out after 3 seconds, so the breaker opens for another 30 seconds.

4. 4

   8:30:06, closed again

   The gateway recovered at 8:30. The next trial succeeds in 400 milliseconds, the breaker closes and card payments resume.

At the worst moment, 6 threads per server were waiting on the gateway, 36 of 300 in total. Product pages and search kept their threads and stayed up.

A circuit breaker should trip when failures are frequent enough that the next call will probably fail too, not on one unlucky error. You decide that with a few settings.

There are two common ways to count. A consecutive count trips after a number of failures in a row, like the shop's 5. A failure rate trips when, say, half of the recent calls failed.

Rate-based breakers suit busy services. With a consecutive count, one lucky success resets the counter. In a test where half of all calls failed at random, a five-in-a-row breaker needed about 60 calls on average to trip.

A rule of 50 percent over the last 20 calls tripped at about call 25 on average. When 4 in 5 calls fail at random, both rules trip within about 20 calls. Recent calls are tracked in a sliding window, either the last N calls or the calls from the last N seconds.

A minimum number of calls stops the breaker from tripping on tiny samples. Resilience4j offers both window types and the minimum.

Slow calls matter too. Resilience4j can mark a call as slow above a time limit and trip when too many calls are slow. A gateway that answers in 9 seconds hurts you almost as much as one that never answers.

For production, the team later moved from its own code to Resilience4j. The table shows the main settings, the defaults in Resilience4j 2.4.0 source code, and the shop's choices.

Look at the slow call rate row. By default, Resilience4j trips on slowness only when every call in the window is slow. Lower it, or slow calls will rarely trip the breaker on their own.

Defaults are a starting point, not advice for your traffic. Polly 8.8.0 has different ones. It breaks when failures reach about 10 percent of calls within 30 seconds, with at least 100 calls, and stays open 5 seconds.

Not every error should count. Fowler notes that some failures are normal and belong to ordinary logic. A declined card is a correct answer from a healthy gateway, so the shop never counts it.

Timeouts, connection errors and 5xx replies do count. The 5xx codes, 500 to 599, are HTTP's server-error replies: the problem is on the server's side. Resilience4j lets you list which exceptions to record and which to ignore.

When the circuit is open, your app should do the most useful thing it can without the dependency. That plan B is called a fallback.

Hystrix, its documentation says, runs the fallback when the circuit is open, when its thread pool or queue is full, or when the call fails or times out. It also advises fallbacks without any network dependency, so the fallback cannot fail too.

Good fallbacks depend on the feature. Here are common ones:

• Offer another way to pay. The shop shows cash on delivery for orders under 5,000 rupees.
• Save the order as payment pending and send a payment link once the gateway is back.
• Show slightly old data, such as yesterday's prices, when a pricing service is down.
• Quietly hide an extra feature, such as recommended items, instead of breaking the page.

Choose fallbacks with the business, not alone. The shop decided that orders above 5,000 rupees would not get cash on delivery, because returns of costly items were a risk. Those shoppers see a message, and their cart is held for 30 minutes.

Here is where the breaker sits in the shop's checkout. Look at the two arrows leaving the breaker: one to the gateway, one to the fallback.

Timeouts, retries and circuit breakers each handle a different kind of failure, and they work best together. Timeouts limit one call, retries handle a short blip, and the breaker handles a longer outage.

• Timeout: caps how long one call can hold a thread. Without it, the breaker sees only waiting, never a failure.
• Retry: tries a failed call again after a short pause. It fixes a one-second hiccup.
• Circuit breaker: notices that failures keep coming and stops all calls, retries included, for a while.

Retries alone made the sale night worse. Each checkout tried 3 times, so a struggling gateway got up to three times the traffic when it was weakest. Spacing retries out with backoff and jitter is a topic of its own.

A common order is the retry on the outside and the breaker on the inside. Each retry attempt then passes through the breaker, and once it opens, attempts fail instantly. Polly's breaker page discusses this same layout, an inner breaker with an outer retry.

The retry should treat an open-circuit error as final for that request. Waiting one more second will not fix a 30-second break, so the shopper gets the fallback straight away.

One more guard is worth knowing: the bulkhead. A bulkhead gives each dependency its own small pool of threads, so a slow dependency can fill only its own pool. Hystrix uses a separate thread pool per dependency for this reason.

You can build a basic circuit breaker in about 25 lines: count failures, remember when you opened, and refuse calls until the wait is over. Here is the shop's first version, before it moved to a library.

It runs on Python 3.13 with only the standard library. The func passed in is the real gateway call, which must have its own short timeout.

import time

class GatewayError(Exception): pass       # the gateway replied with a server error
class CircuitOpenError(Exception): pass   # the breaker refused the call

class CircuitBreaker:
    def __init__(self, max_failures=5, wait_seconds=30):
        self.max_failures, self.wait_seconds = max_failures, wait_seconds
        self.failures, self.opened_at = 0, None   # None means closed

    def call(self, func, *args):
        if self.opened_at is not None:
            if time.monotonic() - self.opened_at < self.wait_seconds:
                raise CircuitOpenError("gateway circuit is open")  # fail fast
        try:
            result = func(*args)          # closed, or a half-open trial
        except (GatewayError, TimeoutError):
            self.failures += 1
            if self.opened_at is not None or self.failures >= self.max_failures:
                self.opened_at = time.monotonic()   # trip, or trip again
            raise
        self.failures, self.opened_at = 0, None   # any success closes it
        return result

Three parts do the real work. The first check inside call refuses at once while the wait is not over: that is the open state. After the wait, the call goes through as a half-open trial.

In the except branch, the check on opened_at means a failed trial trips the breaker straight back to open. The line near the end closes the circuit after any success.

time.monotonic is a clock that cannot go backwards and is not affected by system clock updates, so a changed server time cannot confuse the wait.

Check it yourself

Paste this after the class. It fakes a gateway that times out, then one that works, using a 2-second wait instead of 30 so the test is quick.

def fake_charge(works):
    if not works:
        raise TimeoutError("no answer within 3 seconds")
    return "paid"

breaker = CircuitBreaker(max_failures=5, wait_seconds=2)
for n in range(1, 8):
    try:
        breaker.call(fake_charge, False)
    except CircuitOpenError:
        print(n, "rejected at once")
    except TimeoutError:
        print(n, "failed after trying")

time.sleep(2)   # the wait ends, so the next call is a half-open trial
result = breaker.call(fake_charge, True)
print("trial:", result, "| state:", "closed" if breaker.opened_at is None else "open")

You should see calls 1 to 5 print failed after trying, calls 6 and 7 print rejected at once, and then trial: paid | state: closed. The whole run takes about 2 seconds.

All 10 do. In a test with 10 threads, every call after the wait reached the hanging gateway. The class has no lock and no limit on trials, so half-open behaves like closed until the first trial fails. In the replay, three checkouts per server reached the gateway in each half-open window, not one.

That is fine for learning, but it is the main reason to use a library. Hystrix and Polly allow one probe, and Resilience4j allows a fixed number.

Using it in checkout

Now the checkout code that uses it. GatewayClient, mark_paid and offer_fallback are made-up helpers. The first sends the charge with a fixed idempotency key per order, the second records the payment, and the third runs plan B.

gateway = GatewayClient(timeout_seconds=3)
gateway_breaker = CircuitBreaker(max_failures=5, wait_seconds=30)

def charge_order(order, attempts=3):
    for attempt in range(attempts):
        try:
            payment = gateway_breaker.call(gateway.charge, order.id, order.amount)
            return mark_paid(order, payment)
        except CircuitOpenError:
            break                              # open: no network call, no waiting
        except (GatewayError, TimeoutError):
            if attempt + 1 < attempts:
                time.sleep(0.2 * 2 ** attempt)  # short backoff, then retry
    return offer_fallback(order)

The breaker is created once and shared by every request on that server, so all checkouts feed the same counts. The 3-second timeout lives in the gateway client. Without it, a hung call would never count as a failure.

The retry loop sits outside the breaker. An open circuit ends the loop at once, with no network call and no waiting. In a test where the gateway always timed out, the first checkout made all 3 attempts.

The second checkout tripped the breaker on its second attempt, and its third attempt was refused at once. Every later checkout went straight to the fallback without touching the gateway.

A circuit breaker can live in your code as a library, or in a proxy that sits next to your service and handles its network traffic. Both are common, and they behave differently.

In code, you use a library such as Resilience4j for Java or Polly for .NET. As of 19 October 2026 their latest releases are Resilience4j 2.4.0 and Polly 8.8.0.

Hystrix's README says it is in maintenance mode, and its final release was 1.5.18. Netflix keeps it for existing apps and said it would use open, active projects such as Resilience4j for new internal ones.

Envoy, a popular open-source proxy, also does circuit breaking, but in a different style. It caps the connections, pending requests, active requests, active retries and connection pools a cluster (a group of upstream servers) may use.

Its documentation says these breakers are on by default with modest values, such as 1024 connections per cluster. It argues that it is nearly always better to fail quickly and apply back pressure as soon as possible.

Envoy's limits are fully distributed and not coordinated: each proxy enforces its own. For retries, it recommends retry budgets. With fixed limits, it says to cap retries hard so sporadic failures can retry but retry volume cannot explode.

Envoy's closest match to a failure-counting breaker is outlier detection. It removes a host that behaves unlike the others from the load-balancing pool, for a time that grows each time it is ejected in a row.

An in-process breaker, like the class above, keeps its state in memory on each server. With 6 servers, the shop has 6 gateway breakers, each tripping on what it sees. In the replay, all six opened within about 9 seconds.

The common mistakes are wrong thresholds, counting the wrong errors, one breaker for too many things, and no fallback. Each one makes the breaker trip too often, or never trip when it should.

• One breaker for everything. If card and UPI payments fail separately, one shared breaker blocks both. Use one breaker per dependency, or per endpoint (one address in an API) when they fail independently.
• No timeout, or a long one. A 30-second timeout means 30 seconds of waiting before the breaker learns anything.
• Counting client errors. Declined cards and bad requests are not signs of a sick gateway.
• Tripping on tiny samples. Without a minimum number of calls, two failures at 3 am can open the circuit.
• Checking the state yourself and skipping the call. Polly's documentation warns that the breaker then never moves to half-open, because it is not an active object.
• No alert. A breaker that opens silently hides an outage from the people who should fix it.
• No test. The team now runs a monthly check in staging, a private copy of the site, that makes a fake gateway hang to prove the breaker opens.

Explain it as a guard on one risky dependency. Say what it protects, how it trips, what users see when it is open, and how it recovers.

1. 1

   Name the risk

   Point to the outside call that can hang, like a payment gateway. Say that waiting threads can take down healthy features.

2. 2

   Set timeouts first

   Give the call a short timeout, so a slow dependency becomes a fast failure your code can count.

3. 3

   Add the breaker

   Describe closed, open and half-open. Name your trip rule, such as 50 percent failures over the last 20 calls.

4. 4

   Say what users see

   Describe the fallback: another payment method, a pending order or a clear message.

5. 5

   Fit it with retries

   Use retries with backoff for short blips, only on safe-to-repeat calls, and let the breaker stop retries during a real outage.

6. 6

   Watch it

   Alert on every state change, and track how long the breaker spends open each week.