Choosing an AI Agent Framework: Plain Loop, Graph-Based or Role-Based

You end up paying for machinery your feature never uses, or you miss the one feature it needed. An ai agent framework comparison should start from what your task must do, not from what other teams use.

Picture a common setup. A small team builds an AI assistant for chartered accountants (CAs), the people who file taxes and audits for businesses. The setup is made up, but the problem is real.

The first feature answers questions, such as which GST (goods and services tax) return a client filed for March. It runs as a short loop of the team's own Python and finishes in seconds.

The second feature replies to GST notices, the letters the tax department sends when filings do not match. The assistant reads the notice, drafts a reply, waits for a senior CA to approve it, and only then files it.

That approval can take two days. The server will restart in between. Someone on the team asks whether they should move everything to an agent framework first, because every meetup talk seemed to use one.

This guide answers that question with tests, not opinions. We installed four popular frameworks at their current releases and ran the same notice task on each, using a stand-in model so no API key was needed.

You need seven words. Each one is explained here in plain terms, and the rest of the guide uses them the same way.

• Model: the AI system that reads text and writes a reply. Your code sends it messages, usually over the network to a paid service.
• Tool: a function the model may ask your code to run, such as client_filings. The model only asks; your code runs it.
• Agent: a program where the model picks the next step in a loop, from the tools you allow.
• State: everything one task knows so far, such as the notice, the draft and the reviewer's decision.
• Checkpoint: a saved copy of the state in a file or database, so a task can stop and continue later from the same spot.
• Human-in-the-loop: a step where the run stops and waits for a person, such as the senior CA's approval.
• MCP (Model Context Protocol): an open standard for offering tools to AI apps, so one tool server works with any app that speaks MCP.

Two more words appear in the table. OpenTelemetry is an open standard for recording what a program did, such as traces, which are records of each step with its timing. A licence says what you may do with the code; all four tested frameworks use MIT, a permissive licence.

Here is one picture for the whole choice. For twenty guests at a family puja you use your own chairs. For three hundred guests you call a tent house, and its full shamiana kit arrives with canopy, poles, lights and tables.

Your own chairs are the plain loop: cheap, familiar, fine for a small gathering. The tent house kit is a framework: it handles the big event, but you take the whole kit and arrange things its way.

Where the picture breaks: a tent house leaves after the event, but a framework stays inside your product. You keep upgrading it, debugging it and reading its defaults for as long as the feature lives.

A plain loop agent is a few dozen lines of your own code. It asks the model what to do, runs the tool the model picked, and repeats until the model answers. It is enough when one agent finishes a task in one sitting.

The question-answering feature is exactly this. A question comes in, the model calls a tool or two, and an answer goes out in seconds. If the server restarts mid-question, the CA simply asks again.

The code below runs on Python 3.11, 3.12 and 3.13 with only the standard library. The fake_model function is a stand-in for your call to a model service, scripted to call one tool and then answer.

from types import SimpleNamespace as Step

def client_filings(period):            # a tool: your code, made-up data
    return f"2 returns found for {period}"

TOOLS = {"client_filings": client_filings}

def fake_model(messages):              # stand-in for a model service
    if len(messages) == 1:
        return Step(kind="tool", tool="client_filings", args={"period": "March"})
    return Step(kind="final", text="Both March returns were filed. " + messages[-1])

def answer(question, ask_model=fake_model, max_turns=6):
    messages = [question]
    for turn in range(1, max_turns + 1):     # hard stop after 6 turns
        step = ask_model(messages)
        if step.kind == "final":
            return f"{step.text} (turns used: {turn})"
        tool = TOOLS.get(step.tool)
        if tool is None:
            messages.append(f"No tool named {step.tool}")
            continue
        messages.append(tool(**step.args))
    return "Could not finish. A CA on the team will reply."

print(answer("Did the client file for March?"))

Run it with python3 plain_loop.py. You should see one line: Both March returns were filed. 2 returns found for March (turns used: 2).

What just happened? The model asked for client_filings, your code ran it, and the result went back into the messages. On the second turn the model answered, so the loop stopped after 2 of its 6 allowed turns.

Three lines carry the design. TOOLS is the full list of actions, so the model cannot call anything else. The range is a hard stop, and the last line hands the question to a person instead of guessing.

The loop stops being enough when work must wait. The notice reply waits days for a senior CA, so its progress must be saved and picked up again by a different process. That is the job frameworks compete on.

It should check how each framework handles control flow, saved state, human approval, tools, tracing, licence and maturity. Feature lists and star counts do not tell you whether your own task will pause and resume safely.

• Control flow: who decides the next step. A graph fixes the arrows in code, an agent loop lets the model choose, and a role-based crew lets agents hand work to each other.
• State and persistence: where progress is saved, how often, and whether a new process can load it.
• Human-in-the-loop: how a run stops for a person and how the answer gets back in.
• Tools and MCP: how you give the model functions, and whether MCP servers plug in directly.
• Observability (tracing): what you can see after a run, such as step history or OpenTelemetry traces, and what is sent anywhere by default.
• Licence: whether you can ship the code in a paid product. MIT allows that if you keep its copyright and licence notice.
• Maturity: how long a stable 1.0 has existed, how often releases land, and how many other packages come with it.

We scored each criterion by running code or reading the package source at the pinned version. Where the docs and the code disagreed, the guide says which is which.

All four can pause for a person and resume later, and all four are MIT-licensed. They differ most in control flow, in where state is saved, and in how much comes with the install.

LangGraph builds an agent as a graph of steps over shared state. CrewAI builds a crew of agents with job titles, plus Flows, which run each step when an earlier step finishes. Pydantic AI centres on one typed agent loop. Microsoft Agent Framework pairs agents with graph workflows.

Read the table as trade-offs, not a ranking. Release counts are final releases on PyPI in the 90 days to 23 October 2026. Package counts are what pip, Python's package installer, resolved on Python 3.13 for Linux. Slim and core are the minimal packages. Full pydantic-ai adds a default set of extras, and full agent-framework adds every optional part of core.

For the notice feature, the deciding row is saved state. LangGraph and Microsoft Agent Framework save progress for you. CrewAI saves a Flow when it pauses for feedback. Pydantic AI hands you the run's conversation and expects you, or a durable runner, to keep it.

You can check which version you have before trusting any row. Run python -c "import importlib.metadata as m; print(m.version('langgraph'))" and expect 1.2.16. Swap in the other package names to check the rest.

Beginners can stop here. You now know what a framework adds, the seven checks, and which framework fits which job. The rest of the guide shows the tested code and the traps engineers hit.

A framework earns its weight when it replaces code that is hard to get right, such as saving progress, resuming after a crash and waiting for people. If your loop needs none of that, a framework adds upkeep without removing work.

Roles cost model calls. With a stand-in model that answers in one turn, a three-role sequential crew in CrewAI 1.15.28 made 3 model calls, and one agent made 1. A real model can take several turns per role.

Research also urges caution. A 2025 study, Why Do Multi-Agent LLM Systems Fail?, annotated over 1,600 traces from 7 popular multi-agent frameworks. It found 14 failure modes in 3 groups, and notes benchmark gains are often minimal.

Signs a framework is earning its weight:

• Tasks wait for people, files or other systems for hours or days.
• A crash halfway would lose real work or file something twice.
• Several features need the same saving, approval and tracing plumbing.

Signs it is not:

• Every task finishes in one short sitting.
• Nobody on the team can say what the framework does at each step.
• You chose it because a talk or a job post mentioned it.

Each one saves the paused task somewhere, stops the process, and later loads the saved task in a new process with the person's answer. They differ in what is saved and in which code runs again.

LangGraph: a graph with a checkpointer

The sample below runs on LangGraph 1.2.16 with langgraph-checkpoint-sqlite 3.1.2. It needs no model at all, because a graph step is any Python function. Save it as notice_graph.py.

# pip install langgraph==1.2.16 langgraph-checkpoint-sqlite==3.1.2
import sqlite3, sys
from typing import TypedDict
from langgraph.checkpoint.sqlite import SqliteSaver
from langgraph.graph import StateGraph, START, END
from langgraph.types import interrupt, Command

class Case(TypedDict, total=False):
    draft: str
    decision: str

def draft_reply(state):
    print("draft_reply ran")
    return {"draft": "Draft: the mismatch is a timing difference."}

def review(state):
    print("review started")              # runs again on resume
    return {"decision": interrupt(state["draft"])}

def file_reply(state):
    print("filing:", state["draft"])
    return {}

g = StateGraph(Case)
g.add_node("draft_reply", draft_reply)
g.add_node("review", review)
g.add_node("file_reply", file_reply)
g.add_edge(START, "draft_reply")
g.add_edge("draft_reply", "review")
g.add_conditional_edges("review", lambda s: "file_reply" if s["decision"] == "approve" else "draft_reply")
g.add_edge("file_reply", END)

saver = SqliteSaver(sqlite3.connect("cases.sqlite", check_same_thread=False))
app = g.compile(checkpointer=saver)
cfg = {"configurable": {"thread_id": "case-42"}}

if sys.argv[1] == "start":
    app.invoke({}, cfg)
    print("paused before:", app.get_state(cfg).next)
else:
    app.invoke(Command(resume="approve"), cfg)

Pause and predict: when the second command resumes the run, which print lines appear? Run python notice_graph.py start, then python notice_graph.py approve, and compare with your guess.

The first command prints draft_reply ran, review started and paused before: ('review',). The second prints review started again, then filing: Draft: the mismatch is a timing difference.

The script writes cases.sqlite in the current folder. Delete that file before you run the pair again, so case-42 starts fresh.

What just happened? The draft step did not run twice, because its result was in the SQLite checkpoint. The review step did run twice, because LangGraph restarts the paused step from its first line.

The LangGraph interrupts page says this plainly, and warns against non-idempotent work before interrupt(). Idempotent means safe to repeat: running it twice has the same effect as running it once.

Microsoft Agent Framework: a workflow with request_info()

In Microsoft Agent Framework 1.23.0, a workflow step calls request_info() and the run ends in the state IDLE_WITH_PENDING_REQUESTS. A new process restores the latest checkpoint and passes the answer through the responses argument.

In our test the drafting step did not run again, and the response handler filed the reply. The checkpoints page says checkpoints are created at the end of each superstep, a round in which ready steps run.

CrewAI: a Flow that waits for feedback

In CrewAI 1.15.28, the human_feedback decorator marks a Flow step for review. By default it waits for console input; a custom provider that raises HumanFeedbackPending pauses the flow instead.

Then kickoff() returns a HumanFeedbackPending object, and the state goes to SQLite by default. A new process calls from_pending() with the flow id, then resume("approve"). In our test the paused step did not re-run.

Pydantic AI: a tool that needs approval

In Pydantic AI 2.59.0, a tool declared with requires_approval=True ends the run with a DeferredToolRequests output, if that type is among the agent's output types. With neither that type nor a registered handler for paused tool calls, the run raised UserError. Nothing is saved for you.

The deferred tools page notes that the message history alone does not show which calls need approval. It says to store the run's conversation, which carries the pending requests. We saved result.conversation to one JSON file with ConversationTypeAdapter.

A second process loaded it and started a new run with conversation= and the approval from build_results(). The tool ran once, then the run finished. Saving the history and pending call ids separately also worked. Durable runners such as Temporal, DBOS or Prefect, tools that save each step of a long job, are optional extras.

We hit four traps that a quick read of the docs would miss. Each was checked on the version named, as of 23 October 2026, so pin your version before relying on any of them.

LangGraph re-runs the paused step and saves asynchronously by default

The re-run is documented, so put side effects after interrupt() or in a later step. Less visible is durability, which sets when checkpoints are written.

In the 1.2.16 source, the default durability mode is async: changes are saved while the next step runs. Pass durability="sync" to invoke() or stream() when each step must be saved before the next one starts.

Microsoft Agent Framework can skip a checkpoint with only a warning

FileCheckpointStorage stores state with Python's pickle, its built-in format for saving objects, and only loads an allow-list of safe types. Our custom Approval request type was not on it. The storage checks on save that it can load the data back, so the pause checkpoint failed.

The run did not fail. The log said the failed checkpoint does not fail the workflow run, and the resume then failed with a RuntimeError saying no pending requests were found. Adding allowed_checkpoint_types=["__main__:Approval"] fixed it.

The allow-list is on the checkpoints page. The warning-only behaviour comes from the 1.23.0 source, in agent_framework/_workflows/_runner.py, and is undocumented as of 23 October 2026.

CrewAI's docstring names an attribute that does not exist

The human_feedback module docstring reads feedback through self.human_feedback. On 1.15.28 that raised AttributeError. The CrewAI docs page uses self.last_human_feedback, which worked.

CrewAI also sends anonymous usage telemetry, data about how the library is used, by default. Its telemetry page says CREWAI_DISABLE_TELEMETRY=true switches it off. Its package metadata also requires Python below 3.14.

Pydantic AI 2.x removed a 1.x keyword

Pydantic AI 1.107.9, the latest 1.x release as of 23 October 2026, accepted Agent(instrument=True) with a deprecation warning. Version 2.59.0 raises TypeError for that keyword.

The 1.x warning names the replacement: capabilities=[Instrumentation()]. On 2.59.0 that produced OpenTelemetry spans in our test, and so did Agent.instrument_all() for every agent.

With tracing off, 2.59.0 also printed a banner suggesting a tracing set-up, once per process in our runs. Setting PYDANTIC_AI_NO_BANNER=1 removed it.

The hidden costs are dependency weight, release churn, defaults you did not choose, and lock-in. Count them before you adopt one, because they last as long as the feature does.

Dependency weight differs a lot. On Python 3.13, crewai 1.15.28 resolved to 137 packages with no extras. That base install already includes two vector databases, stores that search text by meaning, and an MCP client. The full agent-framework 1.23.0 resolved to 209, though its core alone needed 11. As of 23 October 2026, 23 of the 29 agent-framework add-on packages it pulled in were beta releases. Each package is code you trust and must keep patched.

Release churn is real even after 1.0. Pydantic AI moved from 1.0 to 2.0 in about ten months, and the instrument keyword went with it. Pin exact versions and upgrade on purpose, with your resume tests running.

Defaults are someone else's decisions. Check what is sent off your machine, such as CrewAI's telemetry, and what is recorded, such as message content in traces. Microsoft Agent Framework leaves content out unless ENABLE_SENSITIVE_DATA is set.

Frameworks also make it easy to hand an agent many tools. OWASP, a non-profit that publishes security guidance, calls this excessive agency: too much functionality, too many permissions or too much autonomy.

OWASP's advice fits any framework. Give the agent only the tools it needs, with the least access, and require a person's approval for high-impact actions. Check permissions in your own systems, never by trusting the model.

Lock-in is the last cost. Your prompts and tools move easily, but a graph, a crew or a workflow is written in one framework's API. Keep business rules in plain functions, so a switch means rewriting only the wiring.

Explain it as a choice led by requirements. Start with a plain loop, name the requirement that breaks it, and pick the framework whose saving and approval model meets that requirement.

1. 1

   Describe the task

   Say how long it runs, whether it waits for people, and what a crash halfway would cost.

2. 2

   Start with a plain loop

   Sketch the loop with a tool list and a turn limit. Say it is enough for tasks that finish in one sitting.

3. 3

   Name the requirement that breaks it

   For example: approval can take two days, so the run must pause and survive restarts.

4. 4

   Compare on the seven checks

   Control flow, saved state, human approval, tools and MCP, observability, licence and maturity. Name the row that decides it.

5. 5

   Handle the resume path

   Say which code runs again on resume, and how you make filing or emailing safe to run twice.

6. 6

   Name the costs and the test

   Mention dependency weight, release churn and lock-in. Then say you would run the same old cases through each option.