Your human in the loop may be a liability sink

Consider a system that is accurate enough to approve. It processes each case in seconds and sends the result to an experienced employee for final sign-off. In this scenario, the employee remains accountable if the decision is wrong.

Now add the operating detail. The reviewer receives hundreds of cases a day, sees only the system’s summary and is measured on throughput. Rejecting a recommendation requires a written explanation. Accepting it takes one click.

There is a human in the loop. There may not be any meaningful human control.

The reviewer has become a liability sink: a named person who can absorb responsibility for a decision largely shaped elsewhere. Here, “liability sink” means a destination for organisational blame and accountability, not a conclusion about legal liability. This arrangement protects the appearance of oversight more reliably than it protects the decision.

A review step is not control

Das Narayandas’s forthcoming November–December 2026 Harvard Business Review article, Why Even Your Best Employees Override AI Systems That Work, identifies an important management signal. Employees may override a technically strong system because the organisation asks them to trust its recommendation while leaving them personally answerable for the outcome. What looks like resistance can be rational self-protection. ¹

The article argues that authority and accountability need to move together. That is necessary, but managers need a test that reaches further. Passing a recommendation through a person does not establish that the person had a practical opportunity to assess it. Nor does a formal right to override establish that using the right is safe, informed or feasible.

The relevant unit of analysis is the whole oversight arrangement. It includes the model, the evidence shown to the reviewer, the time available, the reviewer’s training, the incentives attached to agreement and disagreement, the escalation path and the organisation’s response when outcomes go wrong.

The right to override is only the beginning

For high-risk systems within its scope, the EU AI Act sets out a detailed human-oversight standard. Article 14 requires providers to enable assigned people to understand a system’s capacities and limitations, remain alert to automation bias, interpret its outputs, disregard or reverse them, and intervene or interrupt where necessary. Separately, Article 26(2) requires deployers to assign oversight to people with the necessary competence, training and authority, as well as the necessary support. Those obligations are enacted but, following Regulation (EU) 2026/1744, are scheduled to apply from 2 December 2027 for Annex III systems and 2 August 2028 for high-risk systems covered through Annex I. ²

Those provisions apply to defined high-risk systems, not every AI tool. They are still a useful management standard. An override button is only one component. Effective oversight requires the conditions in which a person can recognise when to use it.

A Joint Research Centre report released in January 2025 shows why presence alone is insufficient. Its mixed-method experiment involved 1,411 human-resources and banking professionals in Italy and Germany. Participants made hiring and lending choices with advice from either a generic system that produced discriminatory recommendations or a system designed for fairness. Human overseers were equally likely to follow the advice from each. Human review did not remove discrimination from the generic system, while participants’ own biases still affected results with the fairer system. ³

The report used experimental scenarios rather than live employment or lending outcomes, so it does not measure the performance of deployed oversight programmes. It does show that adding a professional decision-maker does not automatically correct a system’s bias. The combined human-and-AI process needs its own controls.

The reviewer must be able to know

A reviewer cannot be accountable for detecting an error that the interface makes difficult to see.

At minimum, the reviewer needs the relevant source evidence, the policy or criteria governing the decision, material uncertainty and a clear indication of what the system did not consider. A confidence score is not a substitute for those elements.

If the organisation expects reviewers to calibrate reliance over time, it must give them outcome feedback. The system may improve while the human becomes more cautious, or the human may become more trusting while the system drifts. Neither change will be visible from a monthly override percentage alone.

NIST’s voluntary AI Risk Management Framework 1.0 says human roles and oversight responsibilities need to be clearly defined and differentiated. It also says that data on how often people overrule AI outputs, and why, may be useful to collect and analyse. For managers, the objective should not be a zero override rate; it should be to learn whether challenges occur when they improve the joint result. ⁴

The reviewer must be able to act

Authority on a process chart can disappear in the workflow.

An employee may technically be allowed to reject a recommendation while facing a throughput target that makes investigation impossible. The escalation route may take longer than accepting a doubtful case. Managers may repeatedly ask why the employee “went against the model”. If an override creates work and agreement removes it, the interface and incentives have already expressed a preference.

A study of 698 professional tennis matches across 35 tournaments found that overall error rates fell after Hawk-Eye review was introduced, but the balance of error types shifted: line umpires became more likely to call balls in that were out. The authors’ structural model attributes part of that asymmetry to the perceived cost of being overruled. ⁵

This is not evidence that office workers will behave like tennis officials. It illustrates a broader design risk: oversight changes the incentives of the person being watched. Managers must examine which error becomes safer for the reviewer, not merely whether an AI system catches mistakes.

Responsibility must follow the design

Organisations often say that the human makes the final decision. That phrase can conceal how little of the decision the human controls.

The model provider shaped the capability and its failure modes. The organisation selected the system, set thresholds, chose the information shown to reviewers, allocated review time and designed the performance measures. A frontline employee may make the last visible click without owning any of those decisions.

Madeleine Clare Elish described the human operator in such systems as a “moral crumple zone”: the part that absorbs blame when a complex automated arrangement fails, even though the operator had limited control over its behaviour. ⁶ A 2026 preprint gives the idea a current experimental counterpart. Across four online scenario experiments involving 1,801 participants in AI-assisted lending contexts, participants attributed an average of about ten additional responsibility points, on a 0–100 scale, to a human paired with AI rather than another human. ⁷

The finding measures perceived responsibility. It does not establish legal liability or practice inside real organisations. It is nevertheless a warning: adding AI may increase rather than dilute the blame directed at the named human decision-maker.

One workable allocation is:

• The system owner is responsible for selection, validation, monitoring and change control.
• The process owner is responsible for the workflow, evidence, staffing, incentives and escalation route.
• The reviewer is responsible only for the evaluative judgment that the design gives them a reasonable capacity to exercise.

If the organisation cannot make those boundaries explicit, “human in the loop” is not a control description. It is an unresolved allocation of blame.

The Human Oversight Capacity Test

Before relying on a review step, test the conditions in which the reviewer is expected to exercise judgment. This is an Andrew’s Intel management instrument, not a validated legal or technical standard.

On a narrow screen, each row is shown as a separate card.

A weak result in one condition can neutralise a formal right to intervene. The purpose is not to produce a decorative score. It is to expose which operating condition management must change before assigning consequential review responsibility.

Measure oversight capacity

Managers should calculate how much review the process actually demands.

Required review capacity = case volume × time for a meaningful check

This is a simple planning heuristic, not a validated formula. The time must reflect what the reviewer is expected to do: inspect evidence, compare the recommendation with policy, consider exceptions, record a reason and escalate where necessary. It is not the time required to click approve.

If required capacity exceeds the time and skill available, management has several honest choices. It can reduce the system’s autonomy, add reviewers, route only uncertain or high-impact cases to people, use statistically designed sampling, or accept more organisational responsibility for automated decisions. Keeping nominal per-case approval without funding the work is not an honest control.

A 2026 peer-reviewed conceptual design paper makes a related distinction between the AI’s operative agency and the human’s evaluative agency. The system may do the work while the person verifies it against external evidence and policy. This “solve and verify” division can be efficient, but only when verification is genuinely easier than solving the case again and the reviewer receives the material needed to challenge the result. The paper is based on literature and document analysis rather than a field trial, so its mechanisms still require testing in each operating context. ⁸

Review the system, not only the individual decision

Per-case sign-off is the most visible form of oversight, but system-level review may add more value.

Managers should examine outcome quality, error distribution, complaints, appeals, override reasons and differences between teams. They should test whether an apparently low override rate reflects excellent recommendations, weak reviewer authority or a workload that makes disagreement impractical. They should also review decisions that people changed after the AI recommendation. Human discretion can correct a model, reintroduce bias or do both in different cases. ³

This wider review changes the role of the human. The person is no longer expected to catch every hidden defect at production speed. Reviewers contribute local context and exception knowledge, while the organisation retains responsibility for whether the combined system works.

The management test

Before calling a process human-supervised, ask a simple question:

Could this reviewer reasonably detect, explain and safely reverse a wrong decision before the harm arrives?

If the answer is no, the reviewer should not carry the full weight of the outcome. Management must change the evidence, time, authority or allocation of responsibility.

Human oversight is valuable when it creates genuine evaluative control. Without that control, the human in the loop is merely the last person the decision passes on its way to becoming somebody’s fault.