Human-in-the-loop override rate: why zero is a warning sign for AI agents

If people approve an agent's work and almost never change it, there are two explanations: the agent is excellent, or the reviewer has stopped reading. From the outside they look identical. That is why, in the autonomy model we have designed for ETA (Enterprise Technology Accelerator), an override rate heading for zero earns the agent nothing. It triggers a demotion.

This post explains the four tiers, what has to be true before an agent moves up, what sends it back down, and why we think approval rate is the wrong number to watch. It is a designed mechanism. It isn't an industry standard, and we have no ETA pilot data to show it working yet.

What are the four autonomy tiers?

Two phrases in that table carry most of the weight. The first is per step class. Drafting a summary of an invoice query and releasing a payment are both steps, but they don't share a tier, and a good record on the first earns nothing on the second. The second is a named person. Someone's name sits against every promotion.

Why is approval rate the wrong number?

Because approval rate rewards a reviewer for clicking approve. A reviewer who is busy, trusting or rushed produces an unbroken run of approvals for the same reason a careful one does on an easy week. You can't tell them apart.

Override rate asks a sharper question: how often did a human look at what the agent proposed and change it? It gives the reviewer's attention a signal. It also has a floor. If a competent person reviews a meaningful volume of real decisions and never changes one, either the step is trivial, in which case it belongs at a higher tier on its merits, or the review has become a formality. An override rate trending to zero is an alarm, not a success: it may mean the reviewer stopped reading. That is our view as a company, and it is the rule the design is built on.

What has to be true before an agent is promoted?

All of the following, together. None is sufficient alone, and a single failure holds the agent where it is.

1. Enough decisions to measure. A handful of clean runs says nothing.
2. Error below an agreed threshold, measured against a real outcome and not against the reviewer's opinion.
3. An override rate that is stable and non-zero.
4. Seeded checks caught at the agreed rate. We plant known faults in the work a reviewer sees, to confirm reviewers still catch them.
5. No new failure category since the last review.
6. A named owner signs the promotion with the evidence attached.

The thresholds are set per step class by the named owner before a tier begins. We don't publish default numbers, because a default would be copied without thought, and we would rather the client argued about the figure at the start than discovered it was arbitrary at the end.

What sends an agent back down?

• The error rate breaches its threshold.
• An incident above an agreed severity.
• A change upstream: the process, the model version or the shape of the data.
• The override rate collapses to zero.

The third trigger is the one people argue with. A model upgrade is good news, so why demote? Because the record that earned the tier was earned by a different model. Until the new one has its own record, the old promotion is borrowed trust. That is also why a promotion and a model upgrade can never share a window: if results shift, you need to know which change caused it.

The last trigger is the alarm in the title. It is automatic on purpose. A person who must decide whether to demote an agent that is apparently doing brilliantly will find reasons not to.

What does the zero-override alarm look like in practice?

When it fires, the step class drops a tier. The sequence we intend is to send the next batch of proposals to a reviewer with seeded faults included. If the reviewer catches them, the zero was real: the step was easy and the reviewer was reading, and the class can be promoted again on a fresh record. If the reviewer misses them, the organisation has learned something more valuable than the agent's accuracy.

We apply the same instinct inside software delivery. Review depth in our Harnesses follows risk, and the gates before build name people on the client side, as described in the rules our agents work under. The tiers here extend that idea from one pipeline to cross-functional work.

When does this model not apply?

• Steps with no ground-truth signal. If nothing can say afterwards that the decision was wrong, you can measure agreement but not error, and promotion criteria can't be met honestly.
• Decisions about people. Workflows that decide on hiring, performance or pay should never be a first candidate for the higher tiers, and they carry regulatory duties that tiering doesn't remove.
• Steps that can't be reversed or bounded. They fail our workflow qualification gates, so they aren't candidates for this kind of automation at all.
• Very low volumes. If a step runs a few times a month, you will never gather enough decisions to promote it, and that is fine.

What we don't claim yet

We haven't run this on a client's live workflows. There is no ETA pilot, so no override-rate curve, no promotion record and no result to report. What exists is the design and the reasoning above. Weaknesses we already expect: seeded checks cost reviewer time, thresholds are hard to set before you have data, and a determined reviewer can learn to spot the seeds.

The mechanism sits inside the wider design in the ETA white paper, and sits on top of the measurement described in agents arrived, the silos stayed.

To discuss this model for your own workflows, start with the ETA service page.