Separation of Duties for AI Agents

A few weeks ago, one of my AI agents committed a crime.

It was a small one. While testing its work, the developer agent ran a stray git commit -m init inside its worktree. Empty commit, no changes, no reason. Then my merge step did exactly what I told it to do, brought the branch history across as-is, and a junk commit landed on main with my name on it.

Nothing broke. No tests failed. The reviewer agent had approved the diff, and the diff was fine, because the diff was never the problem. The commit history was.

That bug taught me the thing this whole post is about, so let me start with where the idea came from.

An old control, borrowed

If you have spent any time near finance or audit, you know separation of duties. The person who raises the invoice cannot be the person who approves it. The person who writes the cheque cannot be the person who signs it. Not because anyone is assumed to be a thief, but because a system where one person can do both has no way of catching it when they are.

It is one of the oldest internal controls there is, and it has nothing to do with trust. It is structural. You remove the capability, and then you do not have to rely on anybody's good intentions.

Now look at how most people are using AI to write code in 2026.

The same model writes the implementation, then you ask it to review the implementation. Sometimes in the same conversation. Sometimes in the same message. We have built an entire workflow on the accounting equivalent of "please check your own expenses, and be honest about it."

It works more often than it should. That is exactly what makes it dangerous.

The pipeline

I have been building a tool called emorg. It runs software work the way an engineering org does: a product role writes acceptance criteria, an architect designs, a developer implements, a reviewer reviews, and UAT verifies against the actual ticket. Each one is a separate agent session with its own system prompt and its own tools. Tickets move through gated states in a SQLite state table, and each ticket gets its own git worktree.

The important part is that the state table is the workflow. The orchestrator never moves a ticket except through it, so DONE is unreachable without a recorded PASS at every gate. There is no path where something ships because a model felt good about it.

That was the design. Here is the confession.

"Enforced by tools, not by asking nicely"

That line was in my README for weeks. It was also a lie.

The reviewer had a system prompt telling it not to edit code. It did not have a system preventing it from editing code. The whole thing was running with permissions bypassed, and separation of duties existed entirely at the level of politely worded instructions.

I found out the way you always find out. Not from a test, from litter. Stray PLAN_EM-*.md files sitting in my repository root, outside any worktree, that no human had put there. Agents had been wandering out of their sandboxes for weeks and quietly tidying up after themselves badly enough to leave evidence.

So I rebuilt it properly:

• The reviewer has no Write, no Edit, no Bash. It cannot edit the code it judges, because those tools are not in its list.
• UAT gets Bash and a browser, because verifying a UI means actually running the thing.
• Path guards reject any file operation outside the ticket's own worktree, at the permission layer, before the tool call happens.

The difference between those two versions is the difference between a policy and a control. A policy is a sentence in a prompt. A control is the absence of a capability.

What you do when you cannot remove the capability

Then I added support for other providers, and the neat version fell apart.

The Claude Agent SDK lets me restrict tools per session. Arbitrary agent CLIs do not. If someone wants to run a reviewer through a different vendor's command line tool, I have no way to reach in and take away its file access. The control I had just built could not be applied.

The wrong answer would have been to write a firmer prompt.

What I did instead was let each runner declare whether it can actually enforce read-only. When it cannot, emorg snapshots git status --porcelain before the read-only role runs, and fails the gate if the working tree changed. The agent is still capable of editing files. It just cannot get away with it.

That is a real downgrade and I want to be honest about it. Prevention is better than detection. But detection with a hard failure is the correct next tier, and it is enormously better than a prompt that says please. Security people have known this forever: when you cannot prevent, detect; when you cannot detect, at least log.

While I was there, empirical testing caught something I would never have guessed. Running those CLIs without strict config meant my own personal MCP connectors, Gmail and Figma among them, were silently leaking into pipeline agents. A reviewer agent with access to my email. Nobody wrote that feature. It was just the default.

The gate that earned its keep

The nicest thing that happened was not something I designed.

On one run, the reviewer passed a ticket. Then UAT picked it up, ran the ticket's actual command, and it failed. The work looped back to the developer, who fixed it, and it came through clean.

A static gate missed what a dynamic gate caught. That is the entire argument for separation of duties in three sentences. If the reviewer had also been the one running the code, or if I had trusted a single "looks good to me" from a model that had just written it, that failure ships.

Across a longer run the numbers came out at a 90% first-pass rate, with UAT as by far the dominant cost centre: $6.61 of $11.29 on that batch. UAT is expensive because it does real work. It is also the gate that catches what the cheap gate cannot. I have stopped being annoyed about that line item.

Every mutable surface needs an owner

Which brings us back to the junk commit.

I had covered file state, with tool restriction and tree snapshots. I had covered content, with the diff artifact the reviewer reads. What I had not covered was git history, because it never occurred to me that history was a thing an agent could dirty independently of the files.

Give an agent Bash and a repository, and "review the diff" does not mean "review the history."

The fix was boring: squash-merge ticket branches so exactly one commit lands, authored by the tool, plus a rule that emorg owns git. The lesson is not boring, and it generalises well past my project.

In an agent system, every mutable surface an agent can touch needs an explicit owner. Files, obviously. But also git history, the process table, network egress, your MCP connectors, environment variables, the clipboard, whatever else is sitting there in the ambient environment being quietly writable. If you have not decided who owns a surface, the answer in practice is that any agent that can reach it does.

The thing I keep coming back to is that none of this is really about AI. Everything I ended up building here, separation of duties, gates that cannot be skipped, an append-only record of who approved what, detection where prevention is impossible, is a control framework. Accountants worked this out a long time ago, for the same reason: not because people are bad, but because systems that depend on everyone behaving well are not systems, they are hopes.

We are handing a great deal of autonomy to things that are agreeable, capable, and occasionally wrong in ways that are difficult to see. Prompting them to behave is not a control.

Take the capability away. Then you do not have to wonder.