Sagas: Distributed Transactions Without 2PC

A customer can be charged for an item that never ships. The charge succeeds in one service, the stock check fails in another, and nothing undoes the charge. The saga pattern is a common way to fix this.

Picture a common setup. An online electronics shop once ran checkout as one app with one database. The setup is made up, but the problem is real.

In that app, placing an order was a single database transaction: a group of changes that either all happen or none happen. If any change failed, the database rolled every change back.

Then the team split the app into services. A service is a small program that owns one job and its own database. This style is usually called microservices.

The order service owns orders. The inventory service counts stock. The payment service talks to a payment gateway, an outside company that moves the money. Look at the three databases in the diagram below.

The old checkout code made three calls in a row: create the order, charge the customer, then reserve the item. On normal days, every call worked.

Then came a festival sale. A phone priced at Rs 18,999 sold out in 11 minutes. In those minutes, 37 customers paid and then hit an out-of-stock error at the reservation call.

Their money was taken, and no phone was coming. The code had no rule for undoing a charge, so support staff refunded each customer by hand.

The team wants the system to clean up its own half-finished work, with no support staff involved. The tool for that job is a saga, and the rest of this guide builds one for this checkout.

The saga pattern runs one business action across several services as a chain of local transactions. Each local transaction has a matching compensating transaction that undoes it if a later step fails.

A local transaction is an ordinary transaction inside one service's own database. A compensating transaction is a new transaction that reverses the business effect of an earlier one. Refund reverses charge, and release stock reverses reserve stock.

The idea is older than microservices. Hector Garcia-Molina and Kenneth Salem described sagas in a 1987 Princeton technical report. Their problem was long-lived transactions that held database resources for long periods and delayed shorter ones.

Their fix was to write such a transaction as a sequence of smaller transactions, with other work allowed to run in between. The system then guarantees one of two outcomes.

Either every transaction in the saga completes, or compensating transactions undo the part that ran. In the paper's model, there is no third outcome where the work is left half done.

Written as symbols, a saga has steps T1, T2 and T3, with undos C1, C2 and C3. If T3 fails, the saga runs C2 and then C1. The newest work is undone first.

Compare that with a database transaction. Database people sum it up as ACID: atomic (all or nothing), consistent (rules hold), isolated (no one sees half-done work) and durable (saved changes survive crashes).

A saga keeps the all-or-undone outcome, though it gets there in steps rather than in one go. It gives up isolation between its steps, so other requests can see its half-finished work.

A compensating transaction cancels the business effect of an earlier step by doing new work, not by erasing the old work. A refund does not delete the charge. It adds a second record that cancels it.

Think about shifting house with packers and movers. Packing has unpacking, loading the truck has unloading, and booking the new building's lift slot has cancelling that booking.

Now say the new flat's owner backs out on moving morning. You cannot rewind time, so you cancel the lift slot, unload the truck and unpack, newest step first.

Your home works again, but it is not identical. A few cartons stay taped for a week, and the housing society may keep a cancellation fee. Compensation in software also restores a correct state, not the exact earlier state.

Where the picture breaks: during a house move, nobody else uses your half-packed home. In a saga, other requests can read the half-finished data while the undos run, and every undo must survive being run twice.

Here are the checkout steps, each with its undo:

• Create the order with status PENDING. Undo: mark it CANCELLED with a reason, and keep the row.
• Reserve one unit of stock for this order. Undo: release that unit back to the available count.
• Charge the customer through the gateway. Undo: ask the gateway for a full refund.
• Confirm the order and tell the customer. This step needs no undo, because no step that can fail comes after it.

Notice the first undo. Cancelling keeps the order row instead of deleting it. Support staff, auditors and the customer can all see what happened and why.

A saga runs each step in order, records its progress, and switches to running undos the moment a step fails. The order of the steps matters as much as the undos.

The naive fix keeps the old order (order, charge, stock) and just adds a refund. It works, but now every out-of-stock order charges a customer and then refunds them. So ask which step is hardest to undo.

Go back to the house move. Handing the old flat's keys back to its owner has no real undo, so you do it last, once everything is safely inside the new flat.

In checkout, the charge is the hardest step to undo cleanly: money leaves the customer's account, and getting it back needs the gateway. Reserving stock only changes a number in the shop's own database.

So stock comes first and payment comes last. Microsoft's saga pattern guide calls this point of no return the pivot transaction. Steps after the pivot must be retryable, meaning they are repeated until they succeed.

Here is the new saga as a decision flow. Follow the main line first, then each No branch. A failure path runs only the undos for steps that already happened.

Here is one failed order, start to finish. A customer orders earbuds priced at Rs 2,499, and the bank declines the card.

1. 1

   The order is created

   The order service saves order A2 with status PENDING. Nothing is shown to the customer as confirmed yet.

2. 2

   One unit is reserved

   The inventory service drops the available earbuds from 40 to 39. It records that order A2 holds that unit.

3. 3

   The payment is declined

   The bank declines the card. The payment service saves the failure and reports it back to the order service.

4. 4

   The undo runs

   The order service asks inventory to release the unit held by order A2. The available count goes back to 40.

5. 5

   The order is cancelled

   Order A2 becomes CANCELLED, with the reason payment declined. The customer sees a clear message and was never charged.

A tiny saga runner you can run

Here is that checkout saga in plain Python. Each step is a pair: an action and its undo. The services are stand-in functions that write to a history list, so the file runs on its own.

StepFailed is the error a step raises when a business rule says no, such as a declined card or an empty shelf.

class StepFailed(Exception):
    """A business rule said no, such as a declined card."""

stock = {"earbuds": 40}
history = []

def create_order(o):   history.append("create " + o["id"])
def cancel_order(o):   history.append("cancel " + o["id"])
def release_stock(o):
    stock[o["item"]] += 1
    history.append("release " + o["id"])
def refund_payment(o): history.append("refund " + o["id"])

def reserve_stock(o):
    if stock[o["item"]] == 0:
        raise StepFailed("out of stock")
    stock[o["item"]] -= 1
    history.append("reserve " + o["id"])

def charge_payment(o):
    if o["card"] == "declined":
        raise StepFailed("payment declined")
    history.append("charge " + o["id"])

STEPS = [
    (create_order, cancel_order),
    (reserve_stock, release_stock),
    (charge_payment, refund_payment),
]

def run_saga(order):
    undos = []
    for action, undo in STEPS:
        try:
            action(order)
        except StepFailed as err:
            for earlier_undo in reversed(undos):
                earlier_undo(order)
            return "CANCELLED: " + str(err)
        undos.append(undo)
    return "CONFIRMED"

print(run_saga({"id": "A2", "item": "earbuds", "card": "declined"}))
print(run_saga({"id": "A1", "item": "earbuds", "card": "ok"}))
print(history)
print(stock)

Save it as saga.py and run python3 saga.py. On Python 3.13 it prints exactly this:

CANCELLED: payment declined
CONFIRMED
['create A2', 'reserve A2', 'release A2', 'cancel A2', 'create A1', 'reserve A1', 'charge A1']
{'earbuds': 39}

What just happened? Order A2 reserved a unit, then its card was declined. The runner undid only the finished steps, newest first: release, then cancel. Order A1 went through, so one unit is gone.

The failed step's own undo never runs. A step that says no has changed nothing, and its undo joins the list only after it succeeds. Set the stock to 0 and only cancel runs, which we also checked.

A real runner also saves its progress to a database after every step. Then a restarted runner can resume a half-finished saga instead of forgetting it.

The refund pair never runs here, since payment is the last step that can fail. It is there for the day the team adds a step after payment that can fail, such as booking a courier. Payment then stops being the pivot, and that new step takes its place.

Two-phase commit makes every database hold its changes, locked, and vote before any of them commits. A saga lets each service commit on its own and repairs failures later with undos.

In two-phase commit, or 2PC, a coordinator first asks every participant to prepare. Each one saves its changes durably, keeps its locks and votes yes or no. Only if all vote yes does the coordinator tell everyone to commit.

For this shop, 2PC hits a plain wall. The payment gateway is an outside web API, not a database that takes part in a 2PC protocol.

Some gateways do offer something close. You can authorise a card first, which holds the money, and capture it later. A saga can use that as two steps, with releasing the hold as the undo.

Locks are the other cost of 2PC. On PostgreSQL 16.15, we prepared a transaction that reserved one unit, then closed its session, as if the coordinator had crashed.

We gave the test a 2-second lock timeout. Another buyer's update of that stock row hit it, and hit it again after a full database restart. With PostgreSQL's default lock_timeout of 0, that update simply waits. Plain reads kept working and showed the old count, because a PostgreSQL SELECT does not wait for row locks.

Writes to that row waited until someone ran COMMIT PREPARED, the command that finishes a prepared transaction. During a sale, that is every buyer of that phone.

You can check for this on your own PostgreSQL. Run SELECT gid, prepared FROM pg_prepared_xacts; and any row returned is a prepared transaction still holding its locks.

PostgreSQL ships with this feature off: max_prepared_transactions is 0 by default. Its docs advise keeping it at zero unless an external transaction manager closes prepared transactions promptly.

Neither one always wins. Inside one database, a normal transaction beats both. Two-phase commit suits a few databases that all support it, when short locks are fine. Sagas suit steps that cross services or companies.

In orchestration, one coordinator tells each service what to do next. In choreography, there is no coordinator, and each service reacts to events that the other services publish.

An event is a message saying something already happened, such as StockReserved. A message broker is the system that carries messages between services. Kafka and RabbitMQ are well-known brokers.

• Orchestration: the order service, or a separate orchestrator, holds the list of steps and calls each service. The flow sits in one place, but the orchestrator is one more part that must stay up.
• Choreography: inventory hears OrderCreated and reserves stock, then payment hears StockReserved and charges. No single place shows the whole flow, and services can end up depending on each other in a cycle.

This checkout uses orchestration. It has a strict order, and a missed refund is expensive, so the team wants the plan in one place. The Python runner above is a tiny orchestrator.

You can also run an orchestrated saga on a workflow engine such as Temporal. Its documentation says it records each step in an event history and, after a failure, resumes from the latest recorded state.

You make saga steps safe to retry by making each one idempotent, so running it twice has the same effect as running it once. Then a repeated message or a retry after a crash does no harm.

Retries are normal in a saga. Networks drop replies, and services restart halfway through work. Many message brokers can deliver the same message more than once, which is called at-least-once delivery.

A timeout is not a failure

A declined card is a clear no. A timeout is different: the charge may or may not have happened, and the caller cannot tell which.

So never treat a timeout as StepFailed and rush into undos. Retry the same step with the same idempotency key until the answer is a clear yes or no. Only then move on or start the undos.

Idempotency keys for the refund

The riskiest retry is a refund. The payment service asks the gateway for a refund, the refund goes through, and the service crashes before saving that fact. On restart, it asks again.

An idempotency key fixes this. It is a unique string sent with a request, so the server can recognise a retry of the same request. Stripe, a payments company, documents how its API handles them.

As of 19 October 2026, Stripe's v1 API saves the status code and body of the first request with a key, once the endpoint starts running. A retry with that key gets the saved response, even a saved error.

Stripe rejects a reused key sent with different parameters. It may also prune a key once it is at least 24 hours old, and a key reused after that starts a new request.

Here is the refund undo for this checkout. The names payments_db and gateway are stand-ins for the payments database and the gateway client. The key is built from the order id, so every retry for one order sends the same key.

def refund_payment(order):
    payment = payments_db.get_payment(order.id)
    if payment is None or payment.status != "CHARGED":
        return  # never charged, or already refunded: nothing to undo
    gateway.refund(payment.gateway_ref,
                   idempotency_key=f"refund-{order.id}")
    payments_db.set_status(order.id, "REFUNDED")

The early return makes the undo safe when the charge never happened or was already refunded. The idempotency key makes a repeated gateway call return the first result, as long as the gateway still remembers the key.

We ran this against a fake gateway that remembers keys. A crash before saving REFUNDED, then two retries, sent exactly one refund. When the fake gateway forgot the key between tries, it received two refund calls.

So an undo that may keep retrying for more than a day needs one more guard. Before each late retry, ask the gateway whether a refund already exists for that charge.

Saving data and sending a message together

A second trap hides between the database and the broker. The order service must save the order and also publish a message for the next step. If it saves, then crashes before publishing, the saga silently stops.

The transactional outbox pattern solves this. The service writes the outgoing message into an outbox table, in the same local transaction as the business data. A separate relay process reads the outbox and publishes each message.

Because both rows share one transaction, the message is sent if and only if the transaction commits. The pattern's page also warns that the relay might publish a message more than once, so receivers must be idempotent too.

Here is a duplicate-safe receiver as one PostgreSQL statement. It records the message id, and it changes the stock only if that id is new. Both happen in one statement, so they commit together.

WITH first_time AS (
  INSERT INTO handled_messages (message_id) VALUES ('msg-7f3a')
  ON CONFLICT (message_id) DO NOTHING
  RETURNING message_id
)
UPDATE stock SET available = available - 1
WHERE item = 'earbuds' AND available > 0
  AND EXISTS (SELECT 1 FROM first_time);

Create a handled_messages table with message_id as its primary key, then run the statement twice in psql, PostgreSQL's command-line client. PostgreSQL 16.15 printed UPDATE 1 the first time and UPDATE 0 the second, and the count dropped by one only.

Two deliveries of one new id at the same moment were also safe. The second waited for the first to commit, then changed nothing.

UPDATE 0 can also mean the shelf is empty, and the table keeps only the id. So a real receiver also stores the outcome, such as reserved or out of stock, with the message id. A repeated message then gets the same reply as the first.

The saga pattern does not give you isolation, the I in ACID, which stops transactions from seeing each other's half-done work. Other requests can see a saga's partial results while it runs.

Chris Richardson's saga page on microservices.io names lack of isolation as a main drawback. It says saga developers use countermeasures, which are design techniques that make up for the missing isolation.

Microsoft's saga pattern guide names the problems this causes. Here is what each one looks like in this checkout:

• Lost update: two sagas change the same stock count, and one overwrites the other's change.
• Dirty read: a buyer sees the last phone as sold out, but it is only reserved by an order whose payment then fails.
• Fuzzy read: two steps of one saga read the same count and get different answers, because another saga changed it in between.

The false sell-out is usually acceptable for a shop, since the unit comes back once the undo runs. The lost update is not, so we reproduced it.

We used PostgreSQL 16.15 at its default isolation level, read committed, where each statement sees only committed data. Two sagas each read 40 and each wrote back 39. The final count was 39, not 38. One reservation vanished. Our test ran the read and the write inside one step. Across saga steps it is worse: they are separate transactions, so no isolation level can catch it.

Microsoft's guide also lists countermeasures. Three of them fit this checkout:

• Semantic lock: a status field that tells everyone a saga is in progress. PENDING means mid-saga, so invoices and confirmation emails wait for CONFIRMED.
• Commutative updates: changes that give the same result in any order. Subtract one in the database with available = available - 1 instead of writing back a number read earlier.
• Pessimistic view: order the steps so that the updates others act on happen in retryable steps at the end. Here, confirming the order and emailing the customer come last.

We reran the lost-update test with the update written as available = available - 1 WHERE available > 0. The same two sagas left 38, also when their updates overlapped in time.

With one unit left, the first saga changed one row and the second changed none. The count never went below zero, and the zero rows told the second saga it was out of stock.

Finding sagas that are stuck

Sagas can also get stuck. A service can be down for hours, so an undo keeps retrying. Microsoft's guide warns that compensating transactions might not always succeed.

So list every order still PENDING after ten minutes. In this checkout a healthy saga ends within seconds, so each row is a bug or an outage that a person should look at:

SELECT id, status, created_at
FROM orders
WHERE status = 'PENDING'
  AND created_at < now() - interval '10 minutes'
ORDER BY created_at;

We tested it on three orders: one PENDING for 25 minutes, one PENDING for 2 minutes and one CONFIRMED. Only the 25-minute order came back.

At the next sale, the same phone sold out again. Buyers who missed the last unit saw out of stock before paying, and declined cards released their units with no one stepping in.

Use a saga when one business action changes data owned by several services or outside companies, and each change has a sensible undo. Skip it when one database can hold all the data.

• Good fit: checkout across order, stock and payment, where every step can be cancelled or refunded.
• Good fit: travel bookings that hold a flight, a hotel and a cab, each with its own cancellation rule.
• Poor fit: moving money between two accounts in the same database. One local transaction is simpler and fully isolated.
• Poor fit: steps with no undo at all, unless they can safely go last.

Watch for one warning sign. If almost every action in your app needs a saga, your service boundaries may be wrong. Data that always changes together often belongs in one service.

Explain it as a chain of local transactions with undos, then show you know its weak spots: isolation, retries and step order.

1. 1

   Say why one transaction will not work

   Each service owns its own database, and the payment gateway is outside the company. One ACID transaction cannot span them, and 2PC would hold locks or is not supported.

2. 2

   List the steps and their undos

   Create the order as PENDING, reserve stock, charge payment, then confirm. Pair each step with cancel, release and refund.

3. 3

   Justify the order

   Put cheap undos first and the pivot, the step that is hardest to undo, last. Say that steps after the pivot only retry.

4. 4

   Pick a coordination style

   Choose orchestration or choreography in one sentence, with a reason. Say where saga progress is saved, so a crash can resume.

5. 5

   Make every step idempotent

   Mention idempotency keys, handled message ids and the outbox pattern. Add that a timeout is an unknown outcome, so you retry it before undoing anything.

6. 6

   Admit the isolation gap

   Say that other requests can see partial results. Then name your countermeasures: the PENDING status, relative updates and customer messages sent last.