Debugging AI Agents: Reading Transcripts and Stop Reasons

An AI agent loops forever when nothing in the run tells the model, or your code, that the job is over. Usually a tool is failing quietly, and the loop has no limit of its own.

A model is the AI itself: a program that reads text and writes text back. An agent is a program where the model picks the next step, again and again, until the goal is reached. A tool is an action your program lets the model ask for, such as resetting a VPN profile.

Picture a common setup. A company of about 600 staff runs a helpdesk agent that reads IT tickets and fixes simple problems with tools. The setup is made up, but this bug is common. A VPN is the secure connection staff use to reach office systems from outside.

One morning an accountant raises a ticket: her VPN disconnects every ten minutes. Forty minutes later the ticket still says "working on it". The usage page shows 52 requests to the model and about Rs 420 spent, against about Rs 6 for a normal ticket.

The engineer who built the agent wants to add a line to the prompt, the instructions his program sends to the model. The IT lead asks him to wait and first read what the agent actually did. The rest of this guide follows that reading, the four bugs it finds and the code that fixes them.

You need six words: transcript, round, stop reason, tool result, error note and context window. Each one comes with an example from the VPN ticket.

The model service is the system that runs the model and answers your requests. Engineers also call a transcript the message history or the trajectory.

You start debugging AI agents by reading the transcript one round at a time. In each round, ask what the model asked for, what came back, and why the reply stopped. The bug usually sits in the first round where one of those answers looks wrong.

Think of a hand-knitted sweater with one dropped stitch near the bottom. Every row after it comes out wrong, but the fix starts at the first dropped stitch, not the last bad row. The rows are rounds, and the dropped stitch is the first bad tool result. Where the picture breaks: a model can sometimes recover a few rounds later, so the first odd round is a lead, not proof.

This pattern of think, act and read the result was described in a 2022 paper called ReAct. Its authors report that interleaving reasoning and actions made the agent's steps easier for people to interpret. In the diagram below, follow the arrows out of the stop reason box.

Here are the first rounds the engineer found in the VPN run.

1. 1

   Round 1

   The model asks for get_employee with employee 4417. Stop reason: wants a tool. Result: accountant, standard VPN profile.

2. 2

   Round 2

   The model asks for search_help_articles with "VPN keeps disconnecting". Result: a help article saying a profile reset fixes most drop-outs.

3. 3

   Round 3

   The model asks for reset_vpn_profile for employee 4417. Result: an empty string, with no text at all.

4. 4

   Round 4

   The model asks for check_vpn_status. Result: profile last reset 9 days ago, so the reset did not happen.

5. 5

   Rounds 5 to 52

   The reset and the status check take turns. The stop reason is wants a tool every time, and the run never reaches finished.

The natural answer is to try the reset again. That is exactly what the model did, 24 more times. The model chose sensibly with bad information. The fault was in what the code told it, not in the prompt.

What just happened? Reading round by round, the engineer spots the first wrong-looking line in round 3: a tool result that says nothing. He also notices that the model never replied finished, and his loop had no other way to end. That is two bugs already, and neither lives in the prompt.

A stop reason tells you why the model stopped writing: it finished, it ran out of room, it wants a tool, or a safety filter stopped it. Your own loop adds one more way to stop, the round limit.

These labels come from the model service, not from the words the model wrote. A reply can read like a complete answer even when it was cut off. The output limit is the most tokens your program lets the model write in one reply, usually set in each request. A safety filter is a check that blocks text the service does not allow.

Each service uses its own labels. The table shows labels found in the released Python packages of three major model services, checked in October 2026. Your loop should turn them into a few plain labels of its own.

In the VPN run, every reply stopped with wants a tool, which looks healthy one round at a time. The problem only shows across the whole run: 52 in a row, and not one finished.

Why the stop label alone cannot prove a tool was requested

Not every service has a separate tool label. One of the three packages above has no stop reason that means wants a tool, as of October 2026. There, the label cannot tell you a tool was requested; only the reply itself can.

So act on cut off, blocked and unknown labels first, because tool requests in a cut-off reply may be half-written. Otherwise, decide wants a tool by looking for tool requests inside the reply. One reply can also ask for several tools at once, and your loop must run every one of them.

OpenTelemetry, the open standard for traces, metrics and logs, has draft conventions for recording these labels. Its shared list includes stop, length, content_filter, tool_call and error. The conventions are marked Development, meaning they can still change, as of October 2026.

Every tool call should return a result the model can understand, and a failed call should return a clear error note, never silence. The model can only react to what it reads in the transcript.

The engineer opened his tool runner, the small function that runs whichever tool the model asks for. When reset_vpn_profile raised an error, his code caught it and returned an empty string so the program would not crash. The real error: after a security change, the helpdesk's service account could no longer reset VPN profiles. A service account is a login used by a program instead of a person.

The Model Context Protocol (MCP) is an open standard for connecting tools to models. Its current spec, version 2026-07-28 as of October 2026, separates two kinds of failure. Tool execution errors, such as an API failure, a bad input or a business rule, go back inside the tool result marked isError true.

The spec says clients should pass those execution errors to the model so it can correct itself. A request for a tool that does not exist is a protocol error instead, sent as a plain JSON-RPC error, the standard error format MCP messages use. Clients may pass those to the model too, though the spec expects them to help less.

Here is the fixed tool runner in Python. TOOLS is a dictionary that maps tool names to Python functions, such as reset_vpn_profile. When access is refused, these functions raise PermissionError, an error type built into Python.

def run_tool(name, args):
    if name not in TOOLS:
        return {"is_error": True,
                "text": f"No tool called {name}. Tools you can use: {sorted(TOOLS)}"}
    try:
        result = TOOLS[name](**args)
    except PermissionError:
        return {"is_error": True,
                "text": f"{name} failed: the helpdesk account has no permission. "
                        "Retrying will not help. Hand this ticket to the IT team."}
    except Exception as err:
        return {"is_error": True, "text": f"{name} failed: {type(err).__name__}: {err}"}
    return {"is_error": False, "text": str(result)}

The PermissionError branch is the fix for the VPN bug. It tells the model plainly what failed and that retrying will not help. The first check turns a made-up tool name into a note listing the real tools, much as an MCP client may pass on a protocol error.

To check it, call run_tool("reset_vpn_profile", {"employee_id": 4417}) with the failing tool. You should get is_error True and the text "reset_vpn_profile failed: the helpdesk account has no permission." followed by the retry warning. A call with a wrong input name returns a TypeError note that names the bad input, so the model can fix it.

The four common loop bugs are the same call repeated, hidden tool errors, no stop condition in code, and a transcript that outgrows the context window. The VPN run had all four.

• Same call repeated. The model repeats a call because nothing in the results changes, like the reset and the status check taking turns from round 3 to round 52.
• Hidden tool errors. The code around the tool, often called a wrapper, swallows the error and returns nothing, or says ok when the call failed.
• Missing stop condition. The loop ends only when the model says finished, so if it never says it, the loop never ends.
• Context overflow. Each round adds text until the context window fills. The service may then reject the request, or your code must trim old notes, sometimes including the original ticket.

Research points the same way. A 2025 study, "Why Do Multi-Agent LLM Systems Fail?", labelled failures in 1,642 run records from seven multi-agent frameworks, software for building teams of agents. LLM stands for large language model.

In its prevalence figures, step repetition was the most common of 14 failure types, at 15.7%. Not recognising that the task was done came third, at 12.4%. Those were teams of agents, not one agent, so treat the numbers as a pointer rather than a measure of your system.

The diagram below shows how these bugs fed each other in the VPN run. Start at the reset request and follow the circle back to it.

Context overflow made things worse in a quiet way. By round 40, the engineer's code began trimming the oldest notes to fit, and the original ticket went with them. After that, the model was retrying a reset for a problem it could no longer read.

Even before trimming, a long transcript makes the model's job harder. A 2023 study, Lost in the Middle, tested models on long inputs. They often used facts at the start or end better than facts in the middle. So keep tool results short, and keep the goal and the ticket at the start, where your trimming code never touches them.

You stop endless loops with rules in your code: a round limit, a check for repeated calls, and a planned action for every stop reason. A prompt line like "stop when done" helps, but only code can promise that the loop ends.

Here is the new loop. The helper ask_model sends the notes and the tool list to the model service. It returns a dictionary with a plain stop label, the reply text and a list of tool calls, each with an id, a name and its inputs.

ask_model maps the raw label first, so cut off, blocked and unknown labels always win. Tool labels map to wants_tool, and a finished label also becomes wants_tool when the reply holds tool requests. The helper log_round saves one round. The ticket passed in holds both the goal and the ticket text.

import json
from collections import Counter

MAX_ROUNDS = 12     # normal tickets need 3 to 6 rounds
MAX_SAME_CALL = 2   # a third identical call means the agent is stuck

def call_key(call):
    # sort_keys makes {"a": 1, "b": 2} and {"b": 2, "a": 1} the same key
    return call["name"] + " " + json.dumps(call["args"], sort_keys=True, default=str)

def run_agent(ticket):
    notes = [ticket]    # the goal and the ticket stay first, always
    seen = Counter()
    for round_no in range(1, MAX_ROUNDS + 1):
        reply = ask_model(notes)
        log_round(round_no, reply)
        if reply["stop"] == "finished":
            return "done", round_no, reply["text"]
        if reply["stop"] != "wants_tool":    # cut off, blocked or unknown
            return "hand_off", round_no, f"stopped: {reply['stop']}"
        notes.append(reply)
        for call in reply["tool_calls"]:     # one reply can ask for several
            seen[call_key(call)] += 1
            if seen[call_key(call)] > MAX_SAME_CALL:
                return "hand_off", round_no, f"repeated call: {call['name']}"
            notes.append({"call_id": call["id"], **run_tool(call["name"], call["args"])})
    return "hand_off", MAX_ROUNDS, "round limit reached"

The for loop with MAX_ROUNDS replaces the old while True loop, which ended only when the model said finished. Now every run ends. The team's normal tickets finish in 3 to 6 rounds (made up), so 12 leaves room without allowing 52.

The check on wants_tool sends any cut-off, blocked or unknown reply to a person. The caller turns every hand_off result into a ticket in the IT team's queue, with the reason attached.

The repeat check uses Counter, which comes with Python and counts how often each key appears. The key is the tool name plus its inputs written as JSON, a common text format for data. sort_keys makes the same inputs in any order give the same key. A third identical call means the agent is stuck.

To check the fixes, we replayed the VPN run with a short script playing the model, making the same choices as the real run. These are the results on Python 3.13.

• Old loop and old wrapper: still going at round 52, with 52 wants a tool stops, so the test stopped it as the IT lead did.
• New loop and old wrapper: handed off at round 7 with "repeated call: reset_vpn_profile". The repeat check stops the waste but cannot explain it.
• New loop and new wrapper: round 3 returns the permission note, round 4 asks for escalate_to_it_team, and round 5 replies finished.

Follow the same seven steps every time: get the evidence first, change code second, and touch the prompt last. These are the steps the engineer wrote down after the VPN run.

1. 1

   Get the full transcript

   Find the run by ticket id and open every round, not just the final answer. If the transcript is missing, fix logging first.

2. 2

   Check how the run ended

   Read the last stop reason, then scan for long stretches of the same one, like the 52 wants a tool stops in the VPN run.

3. 3

   Find the first odd round

   Look for the first tool result that is empty, says ok without proof, or disagrees with the round before.

4. 4

   Test the tool by itself

   Run that tool call yourself with the same inputs. In the VPN case, this showed the permission error at once.

5. 5

   Check the length

   Compare the transcript's size in tokens with the context window. If old notes were trimmed, check that the goal and the ticket survived.

6. 6

   Fix in code first

   Add the missing error note, limit or check. Change the prompt only when the model chose badly with good information.

7. 7

   Replay and keep the case

   Run the same ticket again with the fix. Then add it to your set of test tickets so the bug cannot quietly return.

For a fast first guess, start at the top of the chart below: did the run end by itself?

Let a short script find it. Scrolling through 52 rounds by eye is slow, and the first bad round is easy to miss. If each round is saved as one line of JSON, a few lines of Python can find the first empty or failed result and the first repeated call.

The script below reads a log where each line holds the round number, the stop label and that round's calls. Each call has a name, its inputs, its result and an is_error flag.

import json
from collections import Counter

def check_log(path):
    rounds = [json.loads(line) for line in open(path, encoding="utf-8")]
    print(f"{len(rounds)} rounds, stop reasons: {dict(Counter(r['stop'] for r in rounds))}")
    seen, first_bad, first_repeat = Counter(), None, None
    for r in rounds:
        for c in r["calls"]:
            key = c["name"] + " " + json.dumps(c["args"], sort_keys=True)
            seen[key] += 1
            if first_bad is None and (c["is_error"] or not c["result"].strip()):
                first_bad = (r["round"], c["name"], c["result"])
            if first_repeat is None and seen[key] == 3:
                first_repeat = (r["round"], c["name"])
    print("first empty or failed result:", first_bad)
    print("first third identical call:", first_repeat)

Run on the saved VPN log, it prints three lines. Expect "52 rounds, stop reasons: {'wants_tool': 52}", then the first empty result at round 3 from reset_vpn_profile, then the first third identical call at round 7.

Round 3 is the dropped stitch, and round 7 is where the new repeat check would have stopped the run. Those two numbers are what you take into the fix.

What the repeat check misses

The repeat check only catches identical calls. An agent that asks for page 1, page 2, page 3 and so on never repeats a key, so it slips past. In our test, such a run went on until the round limit stopped it at round 12.

That is why both checks exist. The repeat check catches the common stuck pattern early and cheaply, and the round limit catches everything else.

The repeat check can also catch the wrong thing. A tool polled on purpose, such as a status check while a reset applies, is called the same way each time. In our test, a status check polled three times handed off in round 3. Exempt such tools, or count a repeat only when the result is also unchanged.

The key also needs care. An earlier version built it with repr(sorted(args.items())), which sorts only the top level. Two identical calls whose nested inputs were in a different order gave different keys in our test, so the check missed them. json.dumps with sort_keys sorts every level.

Log every round with enough detail to replay it: what went in, what came out, the stop reason, the tool calls, the tool results and the cost. The first version logged only the final answer, which said nothing about the 52 rounds before it.

• Run id, ticket id and round number, so rounds can be put back in order.
• The model version and the prompt version, because a change in either can change behaviour.
• The stop reason exactly as the service sent it, plus your plain label.
• Each tool call's id, name, inputs and result, and whether it was an error.
• Tokens in and out for the round, and the time it took, so cost and slow tools are easy to spot.
• How the run ended: finished, handed off, repeated call or round limit.

OpenTelemetry's draft conventions describe one span per tool call, named execute_tool followed by the tool name. A span is one timed step in a trace, the record of a request as it passes through your system. The span can carry the tool name, the call id and an error type when the call fails.

Tool inputs and results are opt-in in those conventions, because they may hold sensitive data. The same caution applies to your own logs. Staff sometimes paste passwords into IT tickets, so mask secrets before saving and decide who may read transcripts.

OWASP is a non-profit that publishes security guidance, including a top 10 list of risks for LLM applications. Its entry on unbounded consumption suggests timeouts, limits on queued and total actions, and logging resource use to spot unusual patterns. A round limit and a cost line in every log follow that advice.

Explain it as a loop you can see into and stop: every round is logged, every stop reason has a planned action, and the limits live in code.

1. 1

   Describe the loop

   Say that the model replies with a stop reason every round, and your code acts on it.

2. 2

   Handle every stop reason

   Name finished, cut off, wants a tool and blocked, plus unknown labels and your round limit, and what the loop does with each.

3. 3

   Make tool errors visible

   Explain error notes, and why a swallowed error makes an agent repeat itself.

4. 4

   Add hard limits

   List a round limit, a repeat check, a cost cap per run and a hand-off to a person.

5. 5

   Log every round

   Name the fields you save, and how you mask secrets before saving them.

6. 6

   Close the loop

   Turn each bad run into a test case that you replay before every change.