---
title: "Humans on the Loop"
date: 2026-11-08
description: Agents act inside rules written before they run. The Board approves how much discretion an agent is given, and on what terms, and asks whether the rules held.
author: Mario Thomas
canonical: https://mariothomas.com/blog/humans-on-the-loop/
---

On 29 October 2026 the founder of Spawn, a 10-person games company, [explained](https://x.com/jsnnsa/status/2115623382721126826) how the business now runs on a fleet of permanent AI agents. Each agent is given a part of the company and the processes that come with it, and it makes the decisions and takes the actions that part needs: deploying code, triaging bugs, keeping the systems running. On 4 November 2026, at the FT's [Future of AI Summit](https://ai.live.ft.com) in London, IgniteTech's chief executive described how his company uses agents to handle bug reports and feature requests: working out what to do with each one, fixing the bug or building the feature, testing it, and shipping it. All of this was happening while he was on stage talking about it.

AI-native is a phrase that turns up a lot in the conversations I have. These two companies are living it: the technology has been designed around what the business needs, and the people now set the direction, set the rules, decide what good looks like, and step in where the agents cannot go. Nobody in either company approves each step, the agents act on their own inside those rules, and the rules are where the limits of their autonomy sit. What makes this possible is what has been built around the model: the harness, which I described in [The Model Did Not Phone Home](/blog/the-model-did-not-phone-home/) as everything around the model that decides whether its proposals become actions.

## What an agent is

Harnesses are not new. When a person works directly with a model, the person is the harness: deciding what to ask, whether to believe the reply, what to do with the output, and when to stop. [Agentic AI](/blog/agentic-ai-explainer/) moved those decisions into software: a program asks the model what to do next, acts on the answer, and asks again until the work is done or something tells it to stop. The person who used to sit between the model and the real world has stepped out of the loop. On its own, that loop has no rules in it: nothing says what the model may not do, what it may not touch, or when it has to stop, so it cannot be trusted with much. Writing those rules before an agent runs is what turns a loop into something a business can govern.

That is where the harness comes back, as software this time rather than a person. The model supplies the judgement each piece of work needs, reading the request, working out what it means, and deciding what to do about it. The harness carries the judgement people made in advance about the limits of that work: which steps are allowed, what must never be touched, and when to stop and ask. It holds the tools the agent is allowed to use, the credentials that decide which systems it can reach, and the budget it can spend before it must stop. It holds the checks a proposed action has to pass before it happens, and the rule that says when the run ends, when it pauses, and when a person has to be asked. And it keeps the record: what was asked, what was allowed, what was done, and what left the organisation's systems. The model and the harness together are what an agent is.

Creating the harness is not a job for engineers alone. Some of the harness is code: the loop that calls the model, the connections to its tools, and the gates that refuse an action. But what counts as a good result, what the agent must never touch, and when a person has to be asked are written in plain language by someone who knows the business. Only the rules that have to hold whatever the model says are then turned into checks the harness enforces. That was my point at the FT summit's private roundtable: building a harness takes governance and domain expertise as much as it takes software. The people who used to sit between the model and the real world are still there, but *humans in the loop have become humans on the loop*.

The harness also settles who decides what the agent does next. There are three answers. A person can decide every step before the run and write the steps down, so the agent follows a fixed workflow that could be drawn as a flowchart, and the model only does the work inside each step. A person can set the workflow but leave a small model to decide which way each request goes at each point. Or the model can run in a loop, deciding each step itself as it goes, with nothing fixed in advance except the rules in the harness. Who picks the next step is a different question from who says yes to it. The agents on my own machine pick their own next step and still ask me before they do anything that matters, which is a loop with a person in it. At Spawn and IgniteTech the yes has moved into the harness, and a person is asked only when the rules say so. Each of the three gives the machine more of the deciding than the one before, and the first thing a Board needs to know about any agent is where its workflow is fixed, where the model is choosing, and who approved that.

The people who build these systems gave both jobs names in 2026. [Harness engineering](https://arxiv.org/abs/2609.00006) is the design of everything that wraps the model: the limits, the checks, and the record. Graph engineering, [a newer and still unsettled term](https://www.truefoundry.com/blog/graph-engineering-enterprise-guide), is the design of the workflow: the steps, and the order the work moves through them. The graph in the name is that flowchart, not a knowledge graph. Each job answers a question a Board will want to ask: who set the limits on this agent, and who designed the steps it follows.

## When the model picks the next step

A fixed workflow can be read, reviewed, and tested before it is ever used, and management can say who designed it and who approved it. That stays true when small models make the decisions along the way. A decision model reads a request and, instead of an answer in words, returns a score for each of a fixed set of options. The software sends the work to the option with the highest score, provided that score clears a threshold the organisation has set. Two decision models shipped in 2026, from [TypeSafe AI](https://docs.typesafe.ai/concepts/system-one) and [Microsoft](https://commandline.microsoft.com/microsoft-decision-1-model-foundry/). A person still sets the options and the threshold, so a Board governs this the way it governs the approval limits in the finance system: who set them, when they were last reviewed, and what happens to the cases that fall outside them. The harder case is when the model picks the next step.

Where the yes has moved into the harness, the model chooses each step and the harness carries it out, and nobody sits between the two. By the time an action happens, the only approval the agent has had is the one written into the rules before the run began. The tools the agent may use, what it may reach and spend, the checks an action must pass, and the rule that stops it are all that stands between a proposal and an action while the loop runs. The industry draws the line in the same place. Anthropic's [guidance](https://www.anthropic.com/engineering/building-effective-agents) separates workflows, where models and tools follow "predefined code paths", from agents, where models "dynamically direct their own processes", and recommends a stopping condition to keep control, such as a limit on how many times the loop may repeat. OpenAI's [guide](https://cdn.openai.com/business-guides-and-resources/a-practical-guide-to-building-agents.pdf) describes agents as systems that work independently on a user's behalf, and names too many failed attempts and high-risk actions as the points where control goes back to a person. Both are describing the rules, because in a loop nothing else governs the agent while it runs.

Whether the rules are enough is the next question.

## Checks the model cannot argue with

Most of the rules in a harness are written for the model to read. A model can read a rule and still not follow it: if it does, good, and if it does not, the rule was only ever a suggestion. The harness is what makes a rule hold either way, because it checks what the model asks it to do instead of taking the model's word for it. Some rules can be enforced before the model is even asked. If the agent has not been given a tool or a credential, it cannot use them, and it cannot spend beyond its budget, whatever it decides. Other rules can only be checked once the model has said what it wants to do, because only then is there something to check: that a change passes its tests before it ships, or that a payment matches an invoice before it goes out. And some checks can only happen afterwards, by looking at what was done. For anything that cannot be undone, the check has to come before the action, not after. The research bears this out. A 2025 [study](https://arxiv.org/abs/2503.13657) led by the University of California, Berkeley, found that failing to check the work is one of the three main ways multi-agent systems break. Anthropic's own [account](https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents) of running coding agents for hours reports the agents declaring work finished too early until they were given a written definition of done and told to test against it. A test the agent has to pass is a check the model cannot argue with.

So the Board question is not whether the model follows the rules. It is who writes them, which of them are enforced, and whether the record shows they held. The rules are decisions people made before the run: what the agent may reach and spend, what counts as done, and which actions are checked before they happen rather than afterwards. A Board can ask who made those decisions, and when they were last looked at. Where the answer is that nobody did, the agent is running on suggestions. Repetition is a reason to automate, and a loop that is checked on every pass can be trusted with a great deal. **A loop is safe where its work can be checked, not where the task repeats.**

## What the Board approves

The Board does not write the rules, set the thresholds, or design the checks. Management does all of that and is responsible for it. What the Board needs is evidence that those decisions were made intentionally, by named people, for reasons that can be explained and defended. [The Principle of Least Intelligence](/remake/library/#principle-of-least-intelligence), a Remake principle, asks for the least capability that achieves the outcome. Applied to permission, it asks for the least discretion. A loop earns its place where letting the model choose has a clear benefit and each pass can be checked. A loop where a fixed workflow would do the same job is capability bought and exposure accepted for no gain. How much discretion the agent is given, and on what terms, is what the Board approves.

The questions are plain ones, and a director can ask them of any agent without knowing anything about the model inside it. Which steps were fixed in advance, and which does the model choose? Who wrote the rules, and when were they last reviewed? What can it reach, and what can it spend? Which actions are checked before they happen, and what happens when a check fails? What happens when it cannot finish, and who is told? The answers have to exist before the agent runs. Written afterwards, they only describe what the agent did, not what it was allowed to do. And whatever the answers, the Board stays accountable for what the agent does. As I argued in [The Accountability Gap](/blog/ai-agency-accountability/), the decisions and the actions can be handed to the machine and the accountability cannot.

The decision that follows, when it comes, is whether to let the agent do more.

## Before the remit grows

Once an agent is running, the next request is usually to let it do more: another tool, a bigger budget, a decision that a person used to make. Each of those changes the rules it runs inside. Some of those changes sit within what the Board has already delegated to management. Where a change goes beyond that, the same questions are asked again, and this time there is a record to read: what the agent did with the discretion it already had. That record is what the Board looks at before the remit grows.

Where the harness itself runs, and who can get at it, is the subject of the next article. None of this makes the model unimportant; it still has to be good enough for the work. **An agent is as safe as the checks that tell it when to stop, not as clever as the model inside it.**

## The Questions Considered {#questions-considered}

### What is an agent's harness, and what does it hold?

An agent is the model and the harness together. The harness is everything built around the model that decides whether its proposals become actions: the tools it may use, the credentials and budget it holds, the checks a proposed action must pass, the rule that stops a run, and the record of what was asked, allowed, and done, and what left our systems. Building one takes governance and domain expertise as much as software.

### How do we know who decides what an agent does next?

There are three arrangements. A person can fix every step in advance, so the agent follows a workflow that could be drawn as a flowchart. A person can set the workflow while a small model decides which way each request goes. Or the model can pick each step itself in a loop, with nothing fixed except the harness rules. The first question for any agent is where its workflow is fixed, where the model chooses, and who approved that.

### What should our Board ask before any agent runs?

Which steps were fixed in advance, and which does the model choose? Who wrote the rules, and when were they last reviewed? What can it reach, and what can it spend? Which actions are checked before they happen, and what happens when a check fails? What happens when it cannot finish, and who is told? The answers must exist before the agent runs, and whatever they are, the Board stays accountable for what the agent does.

### When can we trust an agent that picks its own next step?

A loop is safe where its work can be checked, not where the task repeats. A tool or credential the agent has not been given cannot be used, and it cannot spend beyond its budget, whatever the model decides. For anything that cannot be undone, the check comes before the action. A 2025 study led by the University of California, Berkeley, found failing to check the work is one of three main ways multi-agent systems break.

### Who writes an agent's rules, and what does our Board approve?

Management writes the rules, sets the thresholds, designs the checks, and is responsible for them. What counts as a good result, what must never be touched, and when a person must be asked are written in plain language by someone who knows the business; only the rules that must hold whatever the model says become enforced checks. The Board approves how much discretion the agent is given, and on what terms.

### How should we decide whether to let an agent do more?

The usual next request is another tool, a bigger budget, or a decision a person used to make. Some changes sit within what the Board has already delegated to management. Where a change goes beyond that, the same approval questions are asked again, and this time there is a record to read: what the agent did with the discretion it already had. That record is what the Board examines before the remit grows.