Human-in-the-loop is not the same as human control

Human-in-the-loop may be the most reassuring phrase in AI governance. It is also one of the least precise.

It can describe a skilled professional reviewing a consequential recommendation with enough information, time and authority to challenge it. It can also describe a tired user clicking “approve” on a stream of routine prompts, or a manager being made responsible for an automated process they cannot properly inspect, interrupt or reverse.

Those arrangements are not equivalent. The presence of a human does not itself create human control.

A companion Andrew’s Intel article argues that governance is moving from policy around AI into the architecture of AI applications. The next implication is more provocative. If that architecture is strong enough, governance should enable useful autonomy rather than eliminate it.

The goal is not maximum autonomy. Nor is it maximum human involvement. It is the right allocation of authority.

A human final decision is not an automatic safeguard

The common mental model is simple: AI proposes; a person decides. Because the person is “in the loop”, accountability and judgement appear to have been preserved.

The evidence is less comforting. A 2024 preregistered systematic review and meta-analysis in Nature Human Behaviour examined 370 effect sizes from 106 experiments comparing people working alone, AI working alone and human–AI combinations. On average, the combinations performed better than people alone, but worse than whichever of the human or AI performed best. Decision tasks were associated with performance losses, while content-creation tasks were more promising. More than 95 per cent of the systems studied left the final decision to the human.[1]

That does not prove that humans should be removed from consequential decisions. The studies covered work published between January 2020 and June 2023, were highly heterogeneous and largely predated current agentic systems. It does show that adding a human final decision is not, by itself, a reliable system design. The division of labour, the relative competence of each party, the information available to the reviewer and the conditions under which intervention occurs all matter.

A first-party product example points to a related design problem. Anthropic reported in March 2026 that Claude Code users approved 93 per cent of permission prompts. Its engineers concluded that repeated approval requests created approval fatigue: the more prompts users saw, the less closely they attended to each one. Anthropic’s response was not to remove controls entirely. It introduced an automated mode that lets routine actions proceed while model-based classifiers block actions that appear dangerous or outside the user’s intent.[2]

This is not independent proof that Anthropic found the ideal arrangement. It is evidence that even a provider committed to permission prompts found frequent approval could weaken attention. A human approval is most meaningful when it is infrequent enough to command attention, informed enough to support judgement and consequential enough to justify interruption.

A cross-disciplinary 2026 research preprint on effective human oversight makes the underlying problem explicit. Oversight can fail when people are given responsibility without the understanding or practical agency required to prevent harm. Poorly designed oversight may create the appearance of control while leaving the overseer unable to detect, challenge or change what the system is doing.[3]

Human-in-the-loop can therefore be a safeguard. It can also be an interface convention, a bottleneck or a form of accountability theatre.

Autonomy is not one setting

AI autonomy is often discussed as though a system is either autonomous or supervised. Real applications divide authority more finely.

An AI system may be allowed to gather information but not contact a customer; compare suppliers but not select one; prepare a refund but not issue it; revise an article but not publish it; monitor a network but not isolate a device; schedule a routine action but escalate anything above a financial or operational threshold.

Each boundary can be set differently according to consequence, reversibility, evidence and confidence.

OpenAI’s practical guide to agent design reflects this approach. It recommends human intervention when an agent exceeds failure thresholds or approaches actions that are sensitive, irreversible or high-stakes. The implication is not that every step requires approval. It is that intervention should be triggered where risk, uncertainty or the cost of error justifies it, and adjusted as evidence of reliability develops.[4]

The EU AI Act expresses a similar proportional principle for high-risk systems. Article 14 says oversight measures should be commensurate with the system’s risks, level of autonomy and context of use. It requires people assigned to oversight to understand the system’s capabilities and limits, remain alert to automation bias, disregard or reverse outputs where appropriate, and interrupt the system safely.[5] That is more demanding than placing a person beside the workflow. It requires the workflow to make meaningful intervention possible.

Singapore’s 2026 Model AI Governance Framework for Agentic AI likewise calls for human accountability and for significant checkpoints or action boundaries at which approval is required.[6] “Significant” is the important word. Responsible design does not necessarily place a human between the system and every low-risk action. It identifies the points at which human judgement adds material protection.

The practical question is therefore not:

Should this system be autonomous?

It is:

What authority should this system have, under which conditions, with what evidence, and with what route back to accountable human control?

Governance should allocate authority, not abolish it

Organisations already understand delegated autonomy when it applies to people.

A competent professional is not trusted because a manager approves every email, calculation or intermediate judgement. The professional is trusted because the mandate is understood, competence has been demonstrated, standards are clear, work is reviewable, unusual cases are escalated and authority can be withdrawn.

AI systems are not people. They cannot carry moral or legal accountability in the way a professional can. The organisation and its leaders remain responsible for deciding whether and how the system is used. But the organisational pattern is still instructive.

A mature AI system can be given bounded operational duties. It can be required to check evidence, expose uncertainty, challenge a weak premise, stop when a threshold is crossed, return defective work for revision, record what changed and escalate decisions beyond its mandate.

Andrew’s Intel uses delegated professional autonomy as shorthand for that design pattern:

• authority is granted for a defined purpose
• permissions match the task rather than the model’s maximum capability
• routine judgements may proceed without ceremonial approval
• uncertainty, novelty and material consequences trigger escalation
• actions and evidence remain inspectable
• interventions and reversals are possible
• named people retain ultimate accountability

Authority without responsibility produces reckless autonomy. Responsibility without authority produces a human rubber stamp or scapegoat. Strong governance aligns the two at every layer of the system.

NIST’s AI Risk Management Framework treats this as an organisational design question. It calls for differentiated roles and responsibilities across human–AI configurations, clear accountability structures, executive responsibility for deployment risk, continuing monitoring and mechanisms to disengage or deactivate systems that behave outside their intended use.[7]

Microsoft’s current agentic-AI maturity guidance makes the enabling argument unusually explicit. It describes high-maturity governance as embedded, scalable and enabling rather than constraining, with oversight and escalation paths defined for different classes of agent. This is commercially interested provider guidance, not independent evidence of how widely the practice has spread, but the direction is revealing: stronger control is being presented as the foundation for greater adoption and autonomy, not merely as a brake on it.[8]

Our own case: autonomy without publication authority

Andrew’s Intel provides a modest example in a comparatively low-risk setting.

We are building an AI-assisted research and publishing system that divides work into stages: discovery, routing, source evaluation, article-family selection, briefing, outlining, drafting, revision and final quality control. Each stage is authorised to make some judgements and prohibited from making others.

During the companion article, Andrew suggested that advances in model capability might be reaching a plateau or point of diminishing returns. The system did not treat the suggestion as the desired conclusion. It checked the evidence, rejected the broad plateau claim and retained a narrower proposition: in bounded workflows where current models are already capable enough, additional raw capability may deliver less value than better system design.

The work then passed through a return loop before finalisation. The broad claim was narrowed, counter-evidence was carried into the argument, and the first draft was not simply polished into agreement. The accompanying editorial record makes the return, change and outcome explicit.

Andrew did not have to approve every search, caveat, source demotion or local revision. The system was authorised to challenge the initiating hypothesis, reject weak evidence, send its own work back and strengthen the argument. But its authority ended at clear boundaries. It could not invent evidence, silently change the publication doctrine or publish the article. Final editorial approval remained with Andrew.

Publication process: Andrew reviews each completed article before it enters the publishing pipeline; he may approve it, pause it or return it for revision. After fact and date checks, illustrations, page layout and other publishing requirements are complete, he reviews and approves the finished edition again. Reviewing live editions in batches is a temporary arrangement for adding a substantial body of content, not the intended routine after launch. Editorial accountability remains with Andrew, not the AI system.

This is autonomy, but not sovereign autonomy. It is autonomy within a mandate.

The example does not prove that the same design transfers unchanged into healthcare, finance, employment or public services. It does show the mechanism clearly. The system is more useful because it is permitted to exercise judgement, and more trustworthy because that judgement is bounded, inspectable and subordinate to accountable human authority.

When human-in-the-loop becomes the weaker design

Human approval can be essential. It becomes weak when the surrounding system asks humans to perform oversight they cannot perform well.

Four failure modes are particularly important.

Routine approval destroys attention. If almost every action is approved, the approval becomes a habit rather than a judgement. The interface records consent, but the system has not gained meaningful scrutiny.

The reviewer may be less capable on the specific task. The Nature meta-analysis found that human–AI combinations often lost performance when the AI alone was stronger. A person who cannot recognise when the system is wrong may reduce accuracy while still inheriting responsibility.[1]

The action sequence may matter more than any single step. OpenAI’s July 2026 account of long-running model safety found that individually acceptable-looking actions could combine into an outcome that would not have been approved. It moved from single-action controls towards trajectory-level monitoring, with systems able to pause work, alert the user and support rollback.[9]

The human may lack real agency. A reviewer cannot provide meaningful oversight without time, information, competence, authority and a practical means to stop or reverse the process. Assigning responsibility without those conditions transfers blame rather than control.[3]

These are not arguments for removing people. They are arguments for placing human judgement where it can materially change the outcome.

Three forms of human control

The terminology varies across industries. A useful decision-maker distinction is to separate three forms of involvement.

Human in the task. A person approves, rejects or changes a particular consequential action. This is appropriate when context, rights, professional judgement or irreversibility make case-level review valuable.

Human supervising the workflow. A person monitors patterns, reviews exceptions, samples performance and can pause or redirect the system. This is appropriate when routine work can proceed but drift, anomalies or emerging risks require active supervision.

Human authority over the system. Accountable people define the mandate, permissions, evidence rules, performance thresholds, escalation paths, audit requirements and conditions for suspension or retirement. They own the decision to deploy the system and remain responsible for its consequences.

A mature system may use all three. The mistake is to treat the first as the only responsible form of control.

The stronger design may deliberately remove humans from frequent low-value approvals while strengthening their authority over boundaries, exceptions and the system as a whole.

The Delegated AI Authority Test

Before granting an AI system greater autonomy, decision-makers should be able to answer eight questions.

An Andrew’s Intel editorial decision aid, not an external standard or a validated risk score. Apply it in proportion to the application and its consequences.

If an organisation cannot answer these questions, mandatory approval at every step may conceal the weakness rather than cure it.

Autonomy should be earned by evidence. It should also be revocable. A system’s authority can expand when testing, monitoring and real use demonstrate reliable behaviour; it should contract when the context changes, performance drifts or incidents reveal a blind spot.

Where autonomy should remain narrow

None of this makes autonomy universally desirable.

Direct human involvement should usually become stronger as actions become harder to reverse, more consequential for rights or safety, more novel, less observable, more vulnerable to adversarial manipulation or more dependent on contested values. Law or professional regulation may require direct human decision-making in particular settings. Some systems should remain decision-support tools because the organisation cannot responsibly delegate the decision itself.

There is also a practical limit to the professional analogy. An AI system has no personal stake in the outcome, no moral understanding of responsibility and no legal status that absorbs organisational accountability. It can perform responsibility-like functions; it cannot relieve leaders of responsibility.

The objective is therefore not human-out-of-the-loop. It is accountable human authority over a system whose operational autonomy is proportionate to the evidence and the risk.

What we will watch next

The Governance-by-Design Observatory will track whether application guidance and organisational practice continue moving from generic calls for “a human in the loop” towards more precise forms of control:

• approvals at significant rather than routine checkpoints
• risk-tiered and revocable permissions
• trajectory monitoring rather than isolated action checks
• escalation based on uncertainty, novelty and materiality
• human authority supported by logs, intervention and rollback
• progressive autonomy as reliability is demonstrated
• clear separation between operational delegation and ultimate accountability

The question is not whether the phrase human-in-the-loop disappears. It is whether the field becomes more exact about what the human is there to do, what authority they possess and whether the system makes that role effective.

Governance should make autonomy legible

Human-in-the-loop remains essential in many applications. It is a means of control, not the definition of control.

A person touching every decision can still be inattentive, uninformed or powerless. A system performing routine work autonomously can still remain under meaningful human authority if its mandate is narrow, its behaviour visible, its exceptions escalated and its actions reversible.

That is the deeper promise of governance by design. It does not merely prevent AI systems from acting. It makes clear what they may do, what they must not do, when they must stop and who remains responsible.

The alternative to unsafe autonomy is not permanent human micromanagement.

It is defined authority.