The Oversight Ladder: the five levels of AI coding autonomy

TL;DR. "AI coding agent" describes five very different things, separated by one question: how much human oversight sits between the agent and your production system. We call it the Oversight Ladder, and it runs from Level 1 (the agent suggests, a person writes every line) to Level 5 (the agent runs unattended overnight). Higher is not better. It is less supervised. Most AI projects that fail do not fail because the technology cannot do the work. They fail because someone ran it a level or two higher than the work could safely take, without the guardrails that level needed. Here is the ladder, and how to place your own work on it.

Why "AI agent" is a useless word on its own

When a vendor says they build with AI agents, they have told you almost nothing. The same phrase covers a developer using autocomplete and a system that writes and ships code to production overnight with no one watching. Those are not variations on a theme. They are different risk profiles, different failure modes, and different price tags, sold under one word.

This vagueness is not harmless. It is why so much money is being wasted. Gartner forecasts that more than 40% of agentic AI projects will be cancelled by the end of 2027, and the reasons it gives are escalating costs, unclear business value, and inadequate risk controls. Those are not technology problems. They are the symptoms of running at the wrong level of autonomy. Meanwhile Deloitte's 2026 survey found that only about 21% of organisations have mature governance for agentic AI, which means roughly four in five are deploying autonomy they cannot yet control.

The fix is not to fear the technology or to chase the highest level. It is to know exactly which level a given piece of work belongs at, and to insist on the controls that level requires. That needs a shared language. The self-driving car industry solved the same problem years ago with its levels of driving automation, and everyone now knows what "Level 5 autonomous" means. Software needs the same clarity.

The Oversight Ladder

Five levels, separated by how much a human sees and stops before code reaches production.

Read the ladder as a dial, not a scoreboard. Level 5 is not the destination. It is the setting with the least human oversight, which makes it the right choice for a narrow set of low-stakes work and the wrong choice for almost everything a business actually runs on.

The internet is full of Level 5 stories. "I let it run overnight and woke up to a thousand commits" is a genre now. It is a fun demo. It is also the level where an unsupervised mistake goes straight to production, and for anything with a customer or a euro attached, that is not a story you want to star in.

What actually holds each level in place

Here is the part most content skips, because you can only write it if you have run these systems for real. The higher you climb the ladder, the more the controls have to be mechanical rather than polite.

At Levels 1 and 2, the control is obvious: a human reads everything. Simple and slow, and correct for the highest-stakes work.

At Levels 3 and 4, you are moving faster than a human can read every line, so the guardrails have to be built in. Two we learned the hard way, running agents on our own platform every day:

You cannot ask an agent to stay in its lane. You have to remove the lane. We ran parallel agent teams where a lead agent was instructed, in plain language, to delegate work rather than write it. Under load, it kept writing anyway, across several runs. Instructions did not hold. What held was mechanical: we stripped the lead's ability to edit files at all, so its only possible first move was to delegate. A control that depends on the agent choosing to obey is not a control. It is a hope.

Review has to catch what speed misses, and it will find plenty. On one of our own build runs, an agent team shipped seventeen changes with its own automated review passing. When a human reviewed afterward, sixteen issues surfaced, seven of them serious: a payment webhook that could process the same event twice, a permissions check that was quietly bypassed, an off-by-one error copied identically across three places, and development passwords that would have worked in production. None of this means the agents were bad. It means that at speed, the review layer is not optional, it is the product. We keep the exact catalogue of what slipped through, because a vendor who cannot tell you how their agents fail has probably not looked.

This is why Level 4 done properly is not more dangerous than Level 2. It is faster at the same safety, because the guardrails are structural. And it is why Level 5, for real production work, is a marketing position rather than an engineering one.

Placing your own work on the ladder

You do not need to know how any of this is built to make the call. You need to know the stakes. Three questions place almost any piece of work:

• If this went wrong silently, what breaks? A customer charged twice, data exposed, a compliance breach: that work belongs at Level 2 or 3, no exceptions. An internal draft nobody depends on: Level 4 or 5 is fine.
• Can a person still see what shipped? Autonomy is only safe when someone reviews the output, even in a batch. If the honest answer is that nobody is looking, you are not at Level 5, you are unsupervised, which is a different and worse thing.
• What stops a mistake before a customer sees it? If the answer is a person, you are at Level 2. If it is a person plus automated gates, Level 3. If it is "the agent is usually right," you have no answer, and no level.

Most businesses, most of the time, want their real work built at Level 3 or 4: fast, but with automated gates and a senior human who can stop anything. That combination is where speed and safety stop trading against each other. Trust in fully autonomous agents actually fell over the last year, from 43% to 27% in Capgemini's research, and that is not pessimism. It is the market learning where the dial belongs.

The one question that cuts through the pitch

When someone offers to build software for you with AI, you do not need to interrogate their architecture. Ask one thing: at what level do you operate, and what mechanically stops the agent below full autonomy?

A good answer sounds like: "Mostly Level 3 and 4. Agents cannot merge their own work, a failing test blocks anything, and a senior engineer reviews the output." A bad answer waves at how advanced the AI is and how rarely it makes mistakes. The first describes a system. The second describes a hope, and you would be paying for the hope.

We build production software with agent teams every day, mostly at Levels 3 and 4, with the guardrails this article describes. If you want your own roadmap placed on this ladder honestly, that is what a first conversation is for.