Agentic Design Patterns: Chaining, Routing, Parallel and Loops

When one big prompt does the whole job, the model must read, plan, price and check in one pass, and nothing checks its work. Agentic design patterns fix this by splitting the job into smaller model calls with plain code around them.

A prompt is the instructions and data you send to a model. The model here is a large language model (LLM), software trained on huge amounts of text that reads text and writes text back. A large model handles hard tasks better, and a small model is faster and cheaper.

Picture a common setup. A small travel company in Udaipur plans short holidays in Rajasthan, and customers ask for trips in a chat window. A trip planner replies with an itinerary (a day-by-day plan) and prices. The setup is made up, but the problem is real.

The team gets about 600 messages a day, and a full plan takes a planner about 25 minutes by hand. So the team builds an assistant that drafts the replies. Its first try is one big prompt holding every rule, the hotel price list and the customer's message.

A typical message reads: "We are 4 people from Delhi, arriving on 20 December. Please plan 3 days in Udaipur. Our budget is Rs 40,000."

The one big prompt writes a friendly reply. But the total comes to Rs 52,000, the plan starts on 21 December, and one hotel is not on the company's list. The reply cannot go to a customer.

The model was asked to do five jobs at once: read the message, plan the days, pick hotels, add up prices and write the reply. Nothing checked any of them. This guide fixes the assistant one pattern at a time and shows what each fix costs.

Every fix is built from model calls, so money and waiting time matter. Here a large call costs about Rs 1 and takes about 3 seconds (made up). A small call costs about Rs 0.10 and takes under a second (made up).

Agentic design patterns are small, reusable shapes for arranging model calls and plain code, so an AI feature is easier to control, test and pay for. Each one fixes one problem that a single big prompt handles badly. Five come up again and again:

• Prompt chaining: fixed steps, one after another, with checks in code between them.
• Routing: sort each request first, then send it down the path built for it.
• Parallelization: run calls at the same time, on different parts of a job (sectioning) or on the same question (voting).
• Orchestrator-workers: one model call decides the subtasks for this request, and other calls do them.
• Evaluator-optimizer loop: one call drafts, another checks, and the draft is retried until it passes or hits a round limit.

In chaining, routing, parallel calls and evaluator loops, your code fixes every possible path. Systems built that way are often called workflows. In an agent, the model picks each next step in a loop, such as which tool to call. Orchestrator-workers sits between: the model picks the subtasks, but your code runs and limits them.

One picture covers all five: the kitchen of a big wedding caterer. Here is how its parts map to the patterns:

• Stations in a line (cut, cook, plate), with the head cook tasting between them, are a prompt chain with gates.
• The person at the door who sends each order to the right counter is the router.
• Three cooks making the dal, rice and sweets at once is sectioning. Three tasters who must agree before a dish goes out is voting.
• The head cook reading the guest list, then deciding which dishes to make and who makes each, is the orchestrator.
• Tasting a dish, sending it back with a note, and giving up after a few tries is the evaluator loop.

Where the picture breaks: a cook remembers the last dish, but a model call does not remember earlier calls. It sees only the text you send it this time. And a head cook rarely invents a dish nobody ordered. A model orchestrator can, so your code must check its plan.

Prompt chaining splits one job into a fixed series of model calls, where the output of each step becomes the input of the next. Between any two steps you can add a gate: a check in plain code that stops the work, or sends it another way, when something is wrong.

The assistant becomes a chain of three calls. The first call returns JSON, a simple text format for data that code can read. Notice the gate in the middle: it is plain code, and it can stop the chain before the costly planning step.

Here is the chain in Python. A real system would call your model provider. This version uses canned answers in place of the model, so it runs as printed on any recent Python 3.

from datetime import date

calls = []  # records the size of every model call
CANNED = {  # stand-in answers, so this file runs without a model provider
    "extract": {"people": 4, "start": "2026-12-20", "days": 3, "budget": 40000},
    "plan": "Day 1 City Palace, Day 2 lake boat ride, Day 3 Sajjangarh fort",
    "reply": "Here is your 3-day Udaipur plan within Rs 40,000.",
}

def ask_ai(task, data, size):
    calls.append(size)
    return CANNED[task]

def check_details(d):  # the gate: plain Python, no AI
    problems = []
    if d.get("people", 0) < 1:
        problems.append("group size")
    try:
        date.fromisoformat(d.get("start", ""))
    except ValueError:
        problems.append("start date")
    if not d.get("budget"):
        problems.append("budget")
    return problems

def plan_trip(message):
    details = ask_ai("extract", message, size="small")  # step 1
    problems = check_details(details)                  # gate
    if problems:
        return "Please confirm your " + ", ".join(problems)
    plan = ask_ai("plan", details, size="large")       # step 2
    return ask_ai("reply", plan, size="small")         # step 3

print(plan_trip("We are 4 people from Delhi, arriving on 20 December..."))
print(calls)
print(check_details({"people": 4, "start": "2026-12-32", "budget": 40000}))

Run it and you get three lines: the reply, then ['small', 'large', 'small'], then ['start date']. So the chain made one small call, one large call and one more small call. The last line shows the gate rejecting 32 December, a date that does not exist.

The gate checks that the group has at least one person, the start date is real and a budget is there. If anything fails, the chain asks the customer instead of letting the model guess. Only checked details reach the costly large-model call.

Where prompt chaining breaks and what it costs

A chain breaks when an early mistake flows into every later step. If step 1 reads 20 December as 2 December, the gate passes it, because 2 December is a real date. The whole plan is then wrong but still sounds confident.

That is why the reply should repeat key details, like the dates and group size, so the customer can catch a misread. Gates catch broken details, not wrong ones that look valid.

A chain also breaks when you split too much. Ten tiny steps mean ten calls, ten places to fail and ten waits. This chain costs about Rs 1.20 and takes up to 5 seconds, against Rs 1 and 3 seconds for the one big prompt.

That small extra cost buys steps you can test on their own, like normal functions. A 2021 study of prompt chaining with 20 people compared it with a single-prompt version of the same tool. Independent raters preferred the chained results most of the time. People also rated chains more transparent and easier to control.

The same study saw people debug odd outputs by testing one step of a chain at a time. That habit is the real gain.

Routing means one step, called a router, looks at each incoming request, gives it a label, and sends it down the path built for that label. It keeps each path simple, and it lets easy requests use a cheaper model.

Not every message is a trip request. The team reads a week of old messages and finds four kinds:

• Quick questions, about 50 percent, like whether breakfast comes with the lake hotel. A small model answers from the company's help page.
• New trip plans, about 30 percent, like the Udaipur message. These go to the prompt chain.
• Booking changes, about 15 percent, like moving a date. These go to fixed steps that call the booking system.
• Complaints and anything unclear, about 5 percent. These go straight to a person.

The router itself is one call to a small model with a short prompt. It costs about Rs 0.10 and answers with just one label. Your code reads the label and picks the path, so the model never jumps to a path you did not build.

Where routing breaks and what it costs

Routing breaks when a request lands on the wrong path. Some messages are two kinds at once, like changing a hotel and also adding a day in Jodhpur. A router forced to pick one label will get some of these wrong.

So give the router a safe "unsure" label that goes to a person. Then test the router on its own, as the engineer section below shows.

Now the cost. Before routing, all 600 messages went through the Rs 1.20 chain, about Rs 720 a day. After routing, 300 quick questions cost about Rs 0.20 each, router included. The 180 trip plans cost about Rs 1.30 each.

The 120 changes and complaints need only the router call, about Rs 12 in all. So the total is roughly Rs 306 a day, about 42 percent of the cost.

A 2024 paper trained routers to choose between a strong, costly model and a weak, cheap one for each question. In its best case, on one test set, a router needed 3.66 times fewer strong-model calls than random routing to close half the quality gap between the two models.

On the paper's other test sets the saving was nearer 1.4 to 1.5 times. Routers trained without data resembling the test questions did little better than random. Measure the gain on your own messages.

Parallelization means running several model calls at the same time and then combining their answers. Use sectioning when a job splits into independent parts. Use voting when one wrong answer is costly, so you ask the same question several times and compare.

Sectioning: different parts at the same time

In the chain, one large call plans transport, hotels and activities together. Once the dates and budget are fixed, these parts barely depend on each other. So the team gives each part its own call and runs all three at once.

You can check the timing yourself without a model. This script uses asyncio, Python's built-in library for waiting on several things at once. It fakes three calls that take 3, 2.5 and 3.5 seconds, and starts them together.

import asyncio, time

async def fake_call(part, seconds):
    await asyncio.sleep(seconds)  # stands in for waiting on a model call
    return part

async def main():
    start = time.perf_counter()
    parts = await asyncio.gather(
        fake_call("transport", 3.0),
        fake_call("hotels", 2.5),
        fake_call("activities", 3.5),
    )
    print(parts, f"{time.perf_counter() - start:.1f} s")

asyncio.run(main())

It prints ['transport', 'hotels', 'activities'] 3.5 s, or a little more on a busy machine. The wait is about the slowest call, while running them one after another takes 9.0 s. Replace the gather call with three awaits in a row and it prints 9.0 s. The results come back in the order you passed them, not the order they finished.

Three large calls cost about Rs 3, against Rs 1 for the single planning call. What the team buys is focus: each call has one job, so fewer details get dropped.

Sectioning breaks when the parts are not really independent. If the hotel is 40 minutes from the old city, the activities need longer travel gaps. The combine step checks travel times in code, and sends any clash back for one more pass.

Voting: the same question several times

Voting runs the same check more than once and goes with the majority, or demands full agreement for risky actions. Here it guards one costly action.

A wrong cancellation can cost a customer real money. So when a message might be a cancellation, the assistant asks a small model the same yes-or-no question three times. If all three say yes, the cancellation steps start. If they disagree, a person decides.

The three calls cost about Rs 0.30. A 2022 paper on self-consistency tested the idea on maths and common-sense question sets. It sampled many reasoning paths for each question and took the most common final answer, which clearly beat a single answer.

Voting breaks when the answers are not independent. Temperature is a setting that controls how random a model's choices are. At or near zero, the same prompt tends to give the same answer three times, so the vote adds little.

Turn some randomness on, or ask in different words. And votes multiply cost: five votes cost five calls.

In orchestrator-workers, one model call, the orchestrator, reads the request and decides which subtasks are needed. Other calls, the workers, do those subtasks, and a final step combines the results. Unlike sectioning, the subtasks are not fixed in your code.

Two weeks later, a harder message arrives. A family wedding in Udaipur needs plans for 12 guests. Five fly in from Mumbai, four drive from Jaipur, three elders need an easy day, and everyone needs rooms near the venue.

The first trip had three fixed parts. This one has a different shape, and the next group will have another. So the team lets an orchestrator choose the parts. The diagram shows one run, where it picks four subtasks.

The four workers run at the same time, each with only the details it needs. A final call merges their answers. Because group bookings are large, a planner approves the plan before it reaches the family.

Where orchestrator-workers breaks and what it costs

Orchestrator-workers breaks in three common ways. The orchestrator can split the job badly, for example forgetting the rooms. It can split too finely, creating twenty tiny subtasks. And the combine step can miss a clash, like a flight landing after the hotel's last check-in.

Your code guards against all three. Before any worker runs, it allows at most 6 subtasks, each of a type the company knows. After the merge, it checks that every arrival fits the hotel check-in time.

This run makes six large calls: one to plan, four workers and one to merge. That is about Rs 6 and about 9 seconds, because the workers run side by side.

An evaluator-optimizer loop has one step that writes a draft and another that checks it against clear rules. If the check fails, the feedback goes back to the writer for another try, up to a fixed number of rounds.

The Udaipur plans still had one problem: some drafts came out over budget. So the team adds a loop at the end of the chain. The first draft totals Rs 46,500 against the Rs 40,000 budget.

The feedback says it is Rs 6,500 over. The second draft swaps the lake-view hotel for a cheaper one and comes to Rs 38,900.

Look at the two checks in the diagram. The budget check is plain code, so it is exact, given the right prices. The review is a model call that checks softer rules, like a daily lunch break and no early starts for older guests.

Here is the loop in Python. Like the chain, it uses stand-ins for the two model calls, so it runs as printed.

DRAFTS = [  # stand-in drafts a large model might return, round by round
    {"items": [{"name": "Lake-view hotel, 3 nights", "price": 30000},
               {"name": "Cabs, boat ride and tickets", "price": 16500}]},
    {"items": [{"name": "Old-city hotel, 3 nights", "price": 22400},
               {"name": "Cabs, boat ride and tickets", "price": 16500}]},
]

def draft_plan(details, feedback, round_number):  # stand-in for a large model
    return DRAFTS[min(round_number, len(DRAFTS) - 1)]

def review_plan(details, draft):  # stand-in for the model review of soft rules
    return {"passed": True, "feedback": ""}

def plan_within_budget(details, max_rounds=3):
    feedback = ""
    for round_number in range(max_rounds):  # limit: at most 3 drafts
        draft = draft_plan(details, feedback, round_number)
        total = sum(item["price"] for item in draft["items"])
        print(f"Round {round_number + 1}: total Rs {total}")
        if total > details["budget"]:  # exact check in plain code
            over = total - details["budget"]
            feedback = f"Over budget by Rs {over}. Cut costs."
            continue
        review = review_plan(details, draft)
        if review["passed"]:
            return draft
        feedback = review["feedback"]
    return None  # the caller hands the trip to a person

plan = plan_within_budget({"budget": 40000})
print("Send to the customer" if plan else "Hand to a planner")

It prints Round 1: total Rs 46500, then Round 2: total Rs 38900, then Send to the customer. Change the budget to 30000 and it prints three rounds, then Hand to a planner.

Two parts matter most. The for loop allows at most 3 drafts, so a stubborn case cannot run up a bill. And the free, exact budget check runs before the paid review call, so over-budget drafts never reach the reviewer.

Where the evaluator loop breaks and what it costs

A 2023 paper called Self-Refine showed the basic idea working. One model wrote an answer, gave itself feedback and rewrote it, with no extra training. Across seven tasks, people and automatic scores preferred the refined answers over single-pass ones.

But the loop needs a real signal. A 2023 study found that models asked to recheck their own reasoning, with no outside feedback, often did not improve. Sometimes they got worse. That is why this loop feeds back real numbers from code.

The cost depends on how often drafts fail. A draft costs Rs 1 and a review costs Rs 1, so a plan that passes first time costs Rs 2. The worst case, three drafts and three reviews, costs Rs 6 and takes about 18 seconds. The Udaipur run above cost Rs 3: two drafts and one review, because the over-budget first draft skipped the review.

Choose the simplest pattern that fixes the problem you actually see. Try one well-written prompt first, because it is the cheapest and fastest option. Add a pattern only when a test shows a failure it fixes. Many real systems end up combining two or three.

The table sums up when each pattern fits and what it adds. Every cost in it comes from the made-up prices above.

The final assistant uses all five, each in one place. A router sorts every message. Trip plans go through a chain whose planning step runs in sections, then an evaluator loop checks the budget. Wedding groups go to orchestrator-workers, and voting guards cancellations.

Errors add up because every step must be right for the chain's answer to be right, and voting helps only as much as the votes are independent. A short script shows both with made-up accuracy numbers.

from math import comb

step_ok = 0.95  # each chain step is right 95% of the time (made up)
print(f"3 steps: {step_ok**3:.1%}, 10 steps: {step_ok**10:.1%}")

vote_ok = 0.90  # each vote is right 90% of the time (made up)
majority = sum(comb(3, k) * vote_ok**k * (1 - vote_ok)**(3 - k) for k in (2, 3))
print(f"Majority of 3 independent votes: {majority:.1%}")

false_yes = 0.05  # one call wrongly says "cancel" (made up)
print(f"All 3 wrongly say cancel, independent: {false_yes**3:.4%}")
print(f"All 3 wrongly say cancel, fully correlated: {false_yes:.0%}")

It prints 85.7% for 3 steps and 59.9% for 10 steps. It also prints 97.2% for a majority of three votes, then 0.0125% and 5% for the cancellation check.

The chain numbers assume any wrong step spoils the answer and that steps fail independently. Even then, ten steps that are each 95 percent right give a right answer only about 6 times in 10. This is why each gate matters: it turns a silent error into a question for the customer.

The voting numbers show the catch. If the three cancellation calls fail independently, all three wrongly saying yes happens about once in 8,000 non-cancellation messages that reach the vote. If they fail together, because the same prompt gives the same answer, the false alarm stays at 5 percent. Real calls sit between those two cases, so measure agreement on saved messages before you trust the vote.

The self-consistency paper made its votes differ on purpose. It sampled answers with temperature set to 0.5 or 0.7. It compared the majority vote against one answer from greedy decoding, which always picks the likeliest next token (a word or part of a word).

Its main results used 40 samples per question, but the authors suggest trying 5 or 10 to get most of the gain. That is a useful starting range when you budget for votes.

Test each one on its own against saved examples with known answers, and look at where it goes wrong, not just how often. For the router, label a few hundred old messages by hand, run the router on them and count the mistakes for each label.

Weigh mistakes by what they cost. A quick question sent to a person wastes a few minutes. A cancellation labelled as a quick question can cost a customer money. Track the costly mix-ups separately, and send more borderline cases to the unsure label until they are rare.

A model reviewer, often called an LLM judge, needs the same care. A 2023 study of models used as judges compared pairs of answers. It found position bias, which means favouring an answer because of where it appears.

With the study's default prompt, all three judges tested leaned toward the answer shown first. Even the most consistent one gave the same verdict for both orders only 65 percent of the time.

The same study padded answers by restating a list in new words, adding nothing new. Two of the three judges preferred the longer answer in about 91 percent of 23 tries. The strongest judge fell for it in about 9 percent.

The fix the authors used is cheap. Ask the judge twice with the two answers swapped, and count a win only if both orders agree. Otherwise call it a tie.

The study used this swap in its later experiments. There, the strongest judge agreed with expert human raters on 85 percent of first-turn votes that were not ties. Experts agreed with each other on 81 percent.

For the evaluator loop, that leads to four rules:

• Give the reviewer a short, specific checklist, not "is this plan good?"
• Move every rule you can into plain code, like budget, dates and the hotel list.
• Cap the rounds, and send anything that fails the last round to a person.
• Log every draft and its feedback, so you can see why the loop gave up.

Explain them by starting from one simple design and adding a pattern only when you can name the problem it fixes. Interviewers care more about your reasons and trade-offs than about pattern names.

1. 1

   Clarify the job

   Ask what requests arrive, how many a day, and which mistakes are expensive. For the travel assistant, a wrong cancellation is far worse than a dull itinerary.

2. 2

   Draw the simplest version

   Sketch one prompt or a short chain first. Then name the failure you expect, such as plans that go over budget.

3. 3

   Match each failure to one pattern

   Mixed requests get a router, risky cancellations get a vote, and budget misses get an evaluator loop.

4. 4

   Show where code holds the limits

   Point to the gates, the unsure label, and the caps on rounds, subtasks and votes. Say who gets the request when a limit is hit.

5. 5

   Give rough numbers

   Estimate cost and wait per request, and say they are guesses. For parallel calls, add up the cost but take the slowest call as the wait.

6. 6

   Explain how you would test it

   Describe a saved set of test messages, a separate test for the router, and logs of how many loop rounds each request used.