Reflection Pattern: Making AI Agents Check Their Own Work

It sends out mistakes that a careful person would catch on a second read. A one-pass tool writes each letter once and shows it, so nothing checks the work. The reflection agent pattern adds that second read inside the app.

Picture a common setup. A small team runs a web app that turns a user's problem, typed in plain words, into a letter. The letter is a consumer complaint or an RTI application, a request for government records under India's Right to Information Act. The setup is made up, but the problem is real.

A user might write that a refund for a broken mixer never came. Another might write that a road repair has been stuck for a year.

The first version sends the user's facts and an instruction to a model, the AI that reads and writes text, and shows whatever comes back. It makes about 2,000 drafts a day at about Rs 0.40 each.

Then the product lead reads 300 drafts by hand. About 18 percent have a problem a careful person would catch. The problems fall into three groups.

• Made-up details. A complaint quotes an invoice number the user never gave. The model filled a gap with something that looked right.
• Vague questions. An RTI draft asks for "all details about the road work". An office can easily reply that the request is unclear.
• Wrong tone. A complaint to a shop threatens a police case that the user never asked for.

Each is a mistake a person finds by reading the letter once more, slowly. So the team asks whether the app can read its own draft first. This guide follows them as they build that check, price it and decide where it pays.

You need seven words. Each gets a plain meaning and an example from the drafting app.

• Model request: one call to the model, with text going in and text coming out. Requests are what you pay for and wait for.
• Draft: the model's first full attempt. Example: a complaint about the broken mixer, written from the user's facts.
• Critique: a review of the draft that lists specific problems. Example: the refund demand has no deadline. A critique that only says "make it better" is not useful.
• Revision: a new draft that fixes the problems a critique found. Example: the same letter, now asking for the refund within 15 days.
• Round: one revision plus the critique that checks it. Each round costs at least two model requests.
• Self-critique: the model judging its own draft with nothing else to check against. Research papers call this intrinsic self-correction.
• Outside feedback: any signal that does not come from the model's own judgement. Examples: a code check that finds an unknown date, a failing test, or a user's correction.

The reflection agent pattern is a loop where a model writes a draft, critiques that draft, and revises it using the critique. It repeats until the critique finds nothing or a round limit is reached.

In pure reflection, the same model plays all three parts: writer, reviewer and editor. No person, test or second system tells it what is wrong.

An everyday picture helps. Think of a family preparing a matrimonial biodata, the one-page profile sent to relatives before a match is discussed. Nobody sends the first version.

Where the picture breaks: a person rereading the next morning comes back with a rested mind. A model rereading its draft gets no new information, unless you hand it some, such as the user's facts.

This loop has a well-known research form. A 2023 method called Self-Refine uses one model as writer, critic and editor, with no extra training or data. It ran the loop for up to four rounds.

Across seven tasks, such as dialogue replies and making code more readable, its authors report about 20 percentage points better task performance on average than one-pass output. That average hides a split.

On maths reasoning the gain was close to zero. For one model they tested, the feedback on maths answers said everything looks good in 94 percent of cases. Treat the average as a reason to test, not a promise.

Here is the loop for one letter. Start at the user's facts and follow the arrow from the revise box back to the critique box. That one arrow is the whole pattern.

The loop makes one request for the draft and one for each critique and revision. A final critique always checks the last revision before the loop stops.

Here is one case from the drafting app. A user in Indore bought a mixer online, it arrived broken, and the seller stopped replying.

1. 1

   Draft

   The model writes a complaint about a Rs 3,499 mixer, delivered broken on 04/08/2026. The draft also adds an invoice number, INV-55821, that the user never gave.

2. 2

   Critique 1

   The critic compares the draft with the user's facts and a checklist. It reports two problems: the invoice number is not in the facts, and the refund demand has no deadline.

3. 3

   Revise 1

   The model rewrites only those two parts. It removes the invoice number and asks for a refund within 15 days.

4. 4

   Critique 2

   The new draft passes every checklist item. The critic replies that it found no problems, so the loop stops.

5. 5

   Result

   Four model requests instead of one. The user waits about 14 seconds instead of 4, and the draft costs about Rs 1.15 instead of Rs 0.40.

Sometimes, for gaps and broken rules, but rarely for its own wrong facts or reasoning. With nothing outside to check against, the critic is the same model that made the mistake.

A 2023 study titled Large Language Models Cannot Self-Correct Reasoning Yet tested exactly this on reasoning tasks. Without outside feedback, every model they tried scored lower after two rounds of self-correction than before, on each reasoning test it was given in their main experiments.

The authors put the cause plainly: the models could not properly judge whether their own reasoning was correct. On one maths test, a model changed right answers into wrong ones slightly more often than the reverse: 8.8 against 7.6 percent of questions.

A 2024 survey of self-correction research reached a similar view. It found no clear success with feedback from prompted models alone, except on tasks unusually suited to it.

The same survey found self-correction works well when reliable outside feedback exists. It also works after large-scale fine-tuning, meaning extra training aimed at this skill.

The team sees the same split. They rerun the 300 reviewed drafts through one round of pure self-critique, with no checklist and no facts shown to the critic.

• Caught well: a missing deadline, a missing subject line, a threatening sentence, an RTI question that asks why instead of asking for a record.
• Caught badly: an invented invoice number. Nothing on the page looks wrong, so the critic usually accepts it.
• Caught badly: an RTI application sent to the wrong office, when the model believes that office handles the subject.

Their fix is to make the critique less pure. They hand the critic things to check against, which the engineer sections below build step by step.

A good critique prompt gives the critic a short checklist, the user's original facts, and permission to find nothing. A vague request such as "improve this" produces vague edits.

A checklist, sometimes called a rubric, turns one big judgement into small yes or no questions. Self-Refine's authors found specific, actionable feedback beat generic feedback in their tests. The team follows five rules.

• Show the original facts. The critic can only flag an invented invoice number if it can see what the user really typed.
• Use a short checklist. The team uses five items, each a mistake found in the 300-draft review.
• Allow an empty answer. Say plainly that no problems is a good result, so the critic is not pushed to invent small complaints.
• Ask for a fixed format, one problem per line, so plain code can read the reply.
• Leave out the drafter's reasoning. Show only the facts and the draft, so the critic does not simply agree with the same steps.

Here is what the critique step sees. Notice that the draft goes to two checkers, and only one of them is a model.

In the code below, ask_ai is any function that sends a task name and some text to your model and returns its reply. The critique function keeps only lines that start with PROBLEM:.

CHECKLIST = [
    "Every date, amount and number comes from the user's facts",
    "Each RTI question asks for a record, not an opinion",
    "The letter names the office or shop it is addressed to",
    "Any demand has a clear deadline in days",
    "The tone is firm and polite, with no threats",
]

CRITIC_PROMPT = """Check the draft against the user's facts and every checklist item.
Write each problem on its own line, starting with PROBLEM:
If nothing is wrong, write: No problems found. That is a good result."""


def critique(draft, facts, ask_ai):
    reply = ask_ai("critique", CRITIC_PROMPT, "\n".join(CHECKLIST), facts, draft)
    problems = []
    for line in reply.splitlines():
        line = line.strip()
        if line.startswith("PROBLEM:"):
            problems.append(line.removeprefix("PROBLEM:").strip())
    return problems

A chatty reply that praises the draft adds nothing to the list, because only PROBLEM: lines count. The checklist lines are the team's real failure groups, not general advice.

A 2023 method called Chain-of-Verification pushes the same idea further for facts. The model drafts, writes fact-check questions about its draft, answers them, and then writes a final answer.

Its best versions answer each question without seeing the draft, so the model cannot simply copy its own mistake. On list questions, about 70 percent of single questions were answered correctly, against about 17 percent of items in the tested model's original lists.

Its authors report fewer hallucinations, meaning false statements given as if true. They also say plainly that the method does not remove them completely.

Put the stop rules in your own code, not in the model's hands. The loop below stops on a clean critique, on a revision that did not reduce the problems, or at the round cap.

import re

DATE = re.compile(r"\b\d{2}/\d{2}/\d{4}\b")


def code_checks(draft, facts):
    return [f"Date {d} is not in the user's facts"
            for d in DATE.findall(draft) if d not in facts]


def write_draft(facts, ask_ai, max_revisions=2):
    draft = ask_ai("draft", facts)
    best_draft, best_problems = None, None
    for revision in range(max_revisions + 1):
        problems = code_checks(draft, facts) + critique(draft, facts, ask_ai)
        if best_problems is not None and len(problems) >= len(best_problems):
            break  # this revision did not help, so stop and keep the best one
        best_draft, best_problems = draft, problems
        if not problems or revision == max_revisions:
            break
        draft = ask_ai("revise", facts, draft, "\n".join(problems))
    return best_draft, best_problems

Two details matter. The code_checks function flags any date written as DD/MM/YYYY that the user never typed, with no model involved. The loop also returns the best draft seen, because a late revision can be worse than an earlier one.

Counting problems is a rough score, since a model critic can phrase or split problems differently between runs. Add a spend cap in rupees per draft in production as a last safety net.

Test the loop without a real model

You can check the loop's logic offline with a stand-in model that returns fixed replies. Run the three blocks in one file with Python 3.9 or later, since removeprefix needs it.

class FakeModel:
    """Stands in for a real model so you can test the loop offline."""

    def __init__(self):
        self.calls = 0

    def __call__(self, task, *texts):
        self.calls += 1
        if task == "draft":
            return "My mixer was delivered broken on 04/08/2026. Invoice INV-55821. Refund me."
        if task == "revise":
            return "My mixer was delivered broken on 04/08/2026. Please refund Rs 3,499 within 15 days."
        draft = texts[-1]  # task == "critique"
        lines = []
        if "INV-55821" in draft:
            lines.append("PROBLEM: Invoice number INV-55821 is not in the user's facts")
        if "within 15 days" not in draft:
            lines.append("PROBLEM: The refund demand has no deadline")
        return "\n".join(lines) or "No problems found."


facts = "Bought a mixer for Rs 3,499. Delivered broken on 04/08/2026. Seller stopped replying."
model = FakeModel()
draft, problems = write_draft(facts, model)
print(draft)
print("Open problems:", problems)
print("Model requests:", model.calls)
print(code_checks("Delivered broken on 05/08/2026.", facts))

Run the file. It should print these four lines on Python 3.13.

My mixer was delivered broken on 04/08/2026. Please refund Rs 3,499 within 15 days.
Open problems: []
Model requests: 4
["Date 05/08/2026 is not in the user's facts"]

What just happened? The critic found two problems, one revision fixed both, and a second critique came back clean. That is four model requests: draft, critique, revise, critique.

The last line shows the code check on its own. A date of 05/08/2026 is flagged because the user typed 04/08/2026. Swap FakeModel for your real model call once the logic passes.

Each round adds at least two model requests, and each request re-sends the draft, so cost and waiting grow with every round. Quality gains usually shrink after the first round or two.

Model services usually charge by tokens, small pieces of text, going in and coming out. A critique reads the facts, the checklist and the draft. A revision reads the facts, the draft and the problem list.

Self-Refine went further and appended every earlier draft and critique to the prompt, so its prompts grew each round. Its authors also report diminishing returns as rounds increase, with early rounds giving most of the improvement.

The team tests four revision limits on the 300 reviewed drafts. Pause and predict before reading the table: will four revisions leave fewer problems than two?

Read the last column first. The first revision removes most problems, and the second helps a little. At four, things get slightly worse in this sample, because late rewrites start changing text that was fine.

The worst case is not the usual case. With a limit of two, 60 percent of drafts pass the first critique, 30 percent need one revision, and 10 percent use both. That 40 percent is more than the 18 percent a person flagged, because the critic also raises small points a person would let pass.

Draft plus one clean critique costs about Rs 0.61. So the average is 0.6 x 0.61 + 0.3 x 1.15 + 0.1 x 1.70, about Rs 0.88 a draft.

Across 2,000 drafts a day, that is about Rs 1,760, against Rs 800 with no reflection. Waiting matters too, so disable the button while the loop runs and show progress, or users may start a second, paid loop.

Reflection is worth it when a mistake is costly, the output leaves your hands, and a checklist can describe what good looks like. Skip it for quick, cheap replies that are easy to undo. These signs say reflection will pay:

• The output is filed, sent or printed, like an RTI application that goes to a government office.
• You can list the common mistakes, so the critic has something concrete to check.
• Users can wait a few extra seconds without giving up.
• A few extra model requests cost far less than a bad letter going out.

Some tasks gain little from a second read. These signs say skip it:

• The user will edit the text right away anyway, as in a live preview.
• The task is facts or maths that the model cannot check alone. Use a real check, such as a test or a lookup, instead.
• Replies must arrive in a second or two.
• You have no way to measure whether reflection helps. Build the measurement first.

The team notices something about timing. Users regenerate and edit drafts many times, but only about 700 of the 2,000 daily drafts get exported for filing. So reflection runs only at export.

Now redo the daily maths. All 2,000 quick drafts cost Rs 0.40 each, so Rs 800. The 700 exports add only the reflection part, about Rs 0.48 each, so about Rs 340.

The total is about Rs 1,140 a day. That is about Rs 625 less than reflecting on every draft, and every filed letter still gets checked.

Because the setup often gives the loop help it would not get in your app. The 2023 study and the 2024 survey point to three traps worth checking before you trust any number, including your own.

Trap 1: the right answer decides when to stop

Some earlier results stopped the loop as soon as the answer matched the known correct answer. So a right answer was never revised into a wrong one.

The 2023 study found that these gains vanish when those correct answers, called oracle labels, are not available. Your app never has them, so test without them.

Trap 2: the first prompt was weak

The same study found some gains came from a weak first prompt. On one writing task, adding the missing instruction to the first prompt beat the self-corrected result outright.

Try this before paying for rounds. Put your checklist into the drafting prompt, measure again, and keep reflection only if it still helps.

Trap 3: no fair baseline

A baseline is the simpler method you compare against, given the same number of model requests. For questions with one right answer, the usual one is self-consistency: ask several times and take the most common answer.

In the 2023 study, a debate set-up, where several model copies argue over a maths answer, did worse than plain voting with the same number of answers. At six answers, voting scored 85.3 percent against 83.2.

Letters have no single right answer to vote on. The closest fair baseline there is to write several drafts in one go and keep the one your checks rank best. The survey also notes that self-correction is often not compared with strong enough baselines.

A limit: small models

Model size matters as well. Self-Refine's authors report that a smaller open model struggled badly with the refine step. The Reflexion paper calls the ability to self-correct an emergent quality of stronger, larger models, meaning it shows up as models grow, not in small ones.

1. 1

   Fix a sample set

   Pick 100 to 300 real cases with known problems, and keep them unchanged between tests.

2. 2

   Run four versions

   One pass; one pass with the checklist in the drafting prompt; several drafts ranked by your checks, at equal requests; and the reflection loop.

3. 3

   Score without the answers

   Let the loop stop on its own rules. Never let the known right answer end a run.

4. 4

   Compare three numbers

   Problems left as a percentage, model requests per draft, and seconds per draft. Keep reflection only where it wins on problems per rupee.

Reflection uses the same model to judge its own draft. An evaluator loop uses a separate checker with clear pass rules, and Reflexion carries written lessons from failed attempts into later tries.

The evaluator loop is covered in our agentic design patterns guide. It puts judging in a separate step, which gives it some of the outside feedback pure reflection lacks.

Reflexion, from a 2023 paper, is about learning across attempts. After a failed try, the agent writes a short lesson in words and stores it in memory. The next attempt reads those lessons first.

On a widely used set of Python coding problems, its authors report 91 percent pass@1. That means the one final answer submitted per problem passed the hidden tests in 91 percent of problems.

There is a catch. Its coding agent wrote its own unit tests, small automatic checks of the code. A wrong test that passes bad code makes the agent submit a wrong answer early.

For the drafting app, Reflexion suggests one cheap upgrade. The team saves the critic's most common problems each week and adds the top ones to the drafting prompt, so first drafts improve.

Explain it as a draft, a critique and a revision. Then spend most of your time on what the critic checks against, how the loop stops, and what it costs.

1. 1

   Start with one-pass output

   Draw the plain version: facts in, one model request, draft out. Name the mistakes it makes, with one concrete example each.

2. 2

   Add draft, critique, revise

   Show the loop and say that the same model plays every part. Point out that it needs no training and is quick to add.

3. 3

   Say what the critic checks against

   The user's facts, a short checklist and code checks. Admit that pure self-critique is weak on facts and reasoning.

4. 4

   Give the stop rules

   No problems, no improvement, a round cap and a spend cap, all enforced in code. Return the best version, not the last.

5. 5

   Do the cost maths

   Requests per round, seconds per round, and the average next to the worst case. Show that most of the gain comes early.

6. 6

   Prove it helps

   Compare against a stronger first prompt and several ranked drafts at equal requests, on a fixed sample set, without using the right answers to stop.