Agent Harness Explained: The Code That Turns an LLM Into an Agent

On its own, a large language model can only write text. The code that lets it look things up, remember and stay inside your rules is called an agent harness.

Picture a common setup. An online shop wants an order-support agent that tells customers where their order is and handles small refunds. The setup is made up, but the problem is real.

Send a bare question to a model and it writes something that sounds right, such as "Your order is on its way". It has never seen the shop's order table. Nothing in the model stops it from approving a Rs 5,400 refund either.

The fix is not a cleverer model. It is code around the model: standing rules, tools that read the order table, notes from past chats and a refund limit checked in code. A loop ties these together, and the whole thing is the harness.

You need four words: agent, tool, system prompt and weights. Each gets a plain meaning and an example from the shop below.

Agent. An agent is a program where a model decides the next step, such as which tool to call, and keeps going until the job is done. A script that always runs the same fixed steps is not an agent.

Tool. A tool is a normal function the model may ask the harness to run, such as lookup_order. The model only writes the request. The harness runs the function and hands the result back.

System prompt. The system prompt is the standing set of instructions placed at the start of every request. It tells the model its role and rules, for example "You are an order-support agent".

Weights. Weights are the numbers inside a model, often billions of them, set during training. A frozen model is one whose weights do not change. This guide keeps the model frozen unless it says otherwise.

One warning about the word itself. In software testing, a test harness is code that runs your tests. That older meaning is different. Here, harness means the code around an agent's model.

An agent harness is everything around a frozen model that lets it act as an agent: instructions, tools, memory, safety checks and the loop that runs them. The model supplies the decisions as text. The harness does everything else.

The Harness^2 paper (read as Harness squared), posted on 9 October 2026, writes this as Agent = Model + Harness. It credits that framing to a July 2026 blog essay, Harness Engineering for Self-Improvement.

The essay itself does not write the equation. It describes the harness as the system around a base model that orchestrates execution. It also calls harness engineering closer to runtime and software system design than to prompt templates.

Harness^2 lists eight parts it can edit, each kept as a file: system prompt, guardrails, memory, skills, tool guidance, scripts, subagents and middleware plugins. The table maps each part onto the shop's agent.

Notice two things the table leaves out. The tools themselves, like lookup_order, sit beside the tool guidance. The loop is missing too: in Harness^2 it comes from an existing open-source coding agent that reads the eight files, and only the files get edited.

A picture to hold on to

The Harness Engineering 101 series, discussed on Hacker News on 19 October 2026, puts it in one line: the LLM is the brain, the harness is the body. The mapping works part by part.

• Brain: the model, which decides what to do next.
• Hands: the tools, which actually touch the order table.
• Notebook: the memory, which holds what happened last time.
• Reflexes: the guardrails, which pull the hand back whatever the brain decided.
• Daily routine: the loop of look, decide, act and look again.

Where the picture breaks: a real brain learns from what its body does. A frozen model learns nothing between calls, so every lesson has to be written into the harness. That gap is exactly what the October papers work on.

The harness runs a loop. It builds a prompt, calls the model, and either returns the model's answer or runs the tool the model asked for. Then it calls the model again with the result.

1. 1

   Build the prompt

   Put the system prompt, any memory notes and the conversation so far into one request.

2. 2

   Call the model

   Send the request. The model replies with either a final answer or a tool request, such as lookup_order for A101.

3. 3

   Check the request

   Before any tool runs, guardrail code checks it. A refund above Rs 2,000 is blocked, whatever the model wrote.

4. 4

   Run the tool

   If allowed, the harness runs the real function and gets a result, such as status shipped.

5. 5

   Add the result

   The tool request and its result are added to the conversation, and the loop goes back to step 1.

6. 6

   Stop

   Stop when the model gives a final answer, or when a step budget runs out, so a confused model cannot loop forever.

Our guide on agentic design patterns covers the different shapes this loop can take. The harness view adds one point: every box except the model is code you own and can change.

Here is a complete harness you can run with plain Python 3: no installs and no model API. A stub function stands in for the model and follows fixed rules, so you get the same output every time.

Read the comments marked Part 1 to Part 5. Parts 1 to 4 are the harness. Part 5 tests two versions of the system prompt against the same stub model.

import json
import re

# Fake order data (made up). Amounts are in rupees.
ORDERS = {
    "A101": {"status": "shipped", "eta": "2 days", "amount": 1200},
    "A102": {"status": "delivered", "eta": "none", "amount": 5400},
}


# Part 1: tools, plus a registry the harness looks names up in.
def lookup_order(order_id):
    return ORDERS.get(order_id, {"error": "no such order"})


def issue_refund(order_id):
    return {"refunded": ORDERS[order_id]["amount"]}


TOOLS = {"lookup_order": lookup_order, "issue_refund": issue_refund}


# Part 2: a guardrail. Plain code, so the model cannot talk its way past it.
def guardrail(tool, args):
    if args["order_id"] not in ORDERS:
        return {"error": "no such order"}
    if tool == "issue_refund" and ORDERS[args["order_id"]]["amount"] > 2000:
        return {"error": "BLOCKED: refunds above Rs 2,000 need a human"}
    return None


# Part 3: memory that survives between conversations.
MEMORY = {}


# Stand-in for a real model API. It follows fixed rules, so the run is repeatable.
def stub_model(messages):
    system, task, last = messages[0]["content"], messages[1]["content"], messages[-1]
    found = re.findall(r"A\d{3}", task) or re.findall(r"A\d{3}", system)
    order_id = found[0] if found else None
    if last["role"] == "tool":
        result = json.loads(last["content"])
        if "error" in result:
            return {"answer": "I cannot do that: " + result["error"]}
        if "refunded" in result:
            return {"answer": f"Refunded Rs {result['refunded']}."}
        return {"answer": f"Order {order_id} is {result['status']}."}
    if "refund" in task:
        return {"tool": "issue_refund", "args": {"order_id": order_id}}
    if "Always call lookup_order" in system:
        return {"tool": "lookup_order", "args": {"order_id": order_id}}
    return {"answer": "Your order is on its way."}  # a confident guess


# Part 4: the loop, with a step budget as a second guardrail.
def run_agent(task, system_prompt, max_steps=5):
    if MEMORY:
        system_prompt += " Last order discussed: " + MEMORY["last_order"] + "."
    messages = [{"role": "system", "content": system_prompt},
                {"role": "user", "content": task}]
    for _ in range(max_steps):
        reply = stub_model(messages)
        if "answer" in reply:
            return reply["answer"]
        tool, args = reply["tool"], reply["args"]
        result = guardrail(tool, args) or TOOLS[tool](**args)
        if args["order_id"] in ORDERS:
            MEMORY["last_order"] = args["order_id"]
        messages.append({"role": "assistant", "content": json.dumps(reply)})
        messages.append({"role": "tool", "content": json.dumps(result)})
    return "Stopped: step limit reached."


# Part 5: same model, two system prompts, one small test set.
TESTS = [("Where is order A101?", "shipped"),
         ("Has order A102 arrived?", "delivered"),
         ("Please refund order A102.", "human")]

V1 = "You are an order-support agent. Be brief."
V2 = V1 + " Always call lookup_order before you answer."

for name, prompt in [("v1", V1), ("v2", V2)]:
    MEMORY.clear()
    passed = 0
    for task, expected in TESTS:
        answer = run_agent(task, prompt)
        passed += expected in answer
        print(f"{name} | {task} -> {answer}")
    print(f"{name} score: {passed}/{len(TESTS)}")

MEMORY.clear()
run_agent("Where is order A101?", V2)
print("next chat:", run_agent("Is my last order there yet?", V2))

Save it as tiny_harness.py and run python3 tiny_harness.py. We ran it on Python 3.13.16 on 21 October 2026.

v1 | Where is order A101? -> Your order is on its way.
v1 | Has order A102 arrived? -> Your order is on its way.
v1 | Please refund order A102. -> I cannot do that: BLOCKED: refunds above Rs 2,000 need a human
v1 score: 1/3
v2 | Where is order A101? -> Order A101 is shipped.
v2 | Has order A102 arrived? -> Order A102 is delivered.
v2 | Please refund order A102. -> I cannot do that: BLOCKED: refunds above Rs 2,000 need a human
v2 score: 3/3
next chat: Order A101 is shipped.

What just happened?

With v1, the stub model answered the two order questions without calling any tool. It guessed "Your order is on its way" without checking. That happens to fit A101 but is wrong for A102, which was already delivered. Neither answer gives the real status, so only the refund test passed.

The refund test passed for a reason worth noticing. The model asked to refund Rs 5,400, and the guardrail code blocked it. A rule written only in the prompt could have been ignored. A check in code runs on every tool call, whatever the model writes.

With v2, one added sentence of tool guidance made the stub call lookup_order first. Both order answers became correct, and the score went from 1 of 3 to 3 of 3. Nothing about the model changed.

The last line shows memory. The second chat never mentions an order number. The harness added "Last order discussed: A101" to the system prompt, so the model could still answer.

Be honest about the stub: we wrote it to obey that sentence every time. A real model follows instructions most of the time, not always. Real harness changes must be measured over many runs, not one.

Two checks you can add yourself. Swap in a model that always asks for a tool, and the loop stops after five model calls and returns Stopped: step limit reached. Tell a model to ignore the refund limit, and the guardrail still blocks it. We ran both.

Because the model sees only what the harness shows it and can do only what the harness lets it do. Change the instructions, the tool text or the memory, and the same frozen model makes different choices.

Engineers already tune harnesses by hand. They read a failed run, rewrite a prompt or a tool description, and try again. Our guides on tool design for AI agents and context engineering cover that manual work in depth.

What is new in October 2026 is automation. A model reads failed runs, proposes edits to the harness files, tests them and keeps the edits that help. The model doing the work can be the same one being harnessed.

Harness^2 reports gains of up to 11.5 points on one workplace-task benchmark from harness changes alone, averaged over three of its own runs. Treat that as the authors' result on their chosen tasks, not a general law.

They all treat the harness as something to search and improve, but they differ in what they change and what they measure. The table compares the four papers driving the buzz, as of 21 October 2026.

Read the third row carefully. Self-HPO is not harness-only: it alternates harness search with reinforcement learning on the model, which is training by trial and reward. Its authors say evolving only one of the two is suboptimal.

How Propose-Probe-Compose tunes a harness

Harness^2 has a problem most real tasks share: there is no answer key to check against. So it compares how the agent behaves under different harnesses, instead of checking answers against known correct results.

The paper calls this task-level recursion. A second level, harnessing-level recursion, turns the accepted edits into experience that improves the tuner itself for future tasks. The paper admits the evidence is imperfect, because runs are random and candidates share the same model's biases.

Why cost is now part of the score

Every probe run costs model calls. Harness^2 lists the extra executions as a limitation. HarnessPro goes further and searches for harnesses that are both better and cheaper.

HarnessPro returns three harnesses instead of one: lite for low cost, heavy for best success and medium in between. It scores candidates with tests written by a model, because it uses no official tests or known answers.

The core idea is established; the size of the gains and the reach of automatic tuning are not. Here is how the claims sort, as of 21 October 2026.

• Established, and checkable in code: an agent wraps its model in code, at least a loop, a prompt and some tools. You can read this in any open-source agent, or in the 93 lines above.
• Reported by several papers, not independently confirmed: changing only the harness changes benchmark scores. Each number comes from the authors' own runs on preprints.
• Opinion: the harness matters more than the model. The Self-HPO authors argue against picking one side, since they found that improving both beat improving either alone.
• Unknown: how many teams tune harnesses automatically in production. We found no public numbers, so we give none.
• Unknown: who coined harness engineering. None of the papers, the blog essay or the Hacker News series we checked names an origin.

Practitioners are using the term too. An InfoQ report on a production-AI panel, published 16 October 2026, put harness engineering in its headline. In it, panellist Zichuan Xiong said teams must evaluate intent at the harness level before pull requests are generated.

It can learn to pass the test instead of doing the job. The Self-Evolving Agent Harness paper says the final judge must sit outside what the tuner may edit. It also warns that protecting the judge alone does not stop this kind of gaming.

Bring it back to the toy. A tuner allowed to edit the 2,000 limit or the TESTS list could raise its score without helping a single customer. So the guardrail and the tests must be read-only to any tuner.

The papers and the Harness^2 code name more risks:

• Bad edits that stick. Self-Evolving Agent Harness warns that injected instructions and temporary workarounds can persist across tasks once accepted.
• Cost. Each round needs extra runs, which Harness^2 lists as a limitation.
• Noisy evidence. One good run can be luck, so compare several runs under matched conditions.
• Leaked answers. The Harness^2 code notes that executor network access is unrestricted, and advises restricting it so models cannot reach benchmark answers (as of 21 October 2026).

Self-Evolving Agent Harness also lists controls: least privilege, isolation, independent validation and records that link each decision to its rule version. Least privilege means each part gets only the access it needs. Our guides on least privilege and agent sandboxing go deeper.

Start by hand, with tests, one change at a time. These steps are our recommendation, drawn from the papers above and from the toy.

1. 1

   Put each part in a file

   Keep the system prompt, tool guidance, guardrail rules and memory format in separate files under version control. Harness^2 also keeps each part as a file, which turns every edit into a readable diff.

2. 2

   Build a small test set

   Turn real failed conversations into tests with an expected outcome, like the TESTS list in the toy. A short list of real failures is a better start than a big generic benchmark.

3. 3

   Change one part at a time

   Edit one file, run every test several times, and compare pass rates. A single run can mislead because model output is random.

4. 4

   Lock the judge and the guardrails

   Make tests, scoring code and guardrails read-only to anyone or anything tuning the harness.

5. 5

   Log every run

   Save each run's messages and tool calls, called a trace, so you can see why a score moved.

6. 6

   Track cost beside success

   Record model calls or tokens per task. A harness that wins by doubling cost may not be a win.

Each step links to deeper guides. Context engineering covers memory and compaction, tool design covers tool text, and coding agent architecture covers sandboxes and checks.

Draw the loop, name the parts, then say which ones you would tune and which you would lock.

1. 1

   Give the one-line definition

   Say Agent = Model + Harness: the model makes decisions in text, and the harness is all the code that feeds it, runs its tools and enforces rules.

2. 2

   Draw the loop

   Prompt builder, model, answer-or-tool decision, guardrail check, tool, result back into history, and a step budget.

3. 3

   Name the parts

   System prompt, tool guidance, memory, guardrails, skills, scripts, subagents and middleware. Say which ones your design needs.

4. 4

   Put safety in code

   Explain that limits like the refund cap are enforced in harness code, not only written in the prompt.

5. 5

   Explain how you would improve it

   Describe a test set, one change at a time, repeated runs and cost tracking. Mention that 2026 papers automate this, with self-reported gains.