Human-in-the-loop

Origin

Thomas Sheridan and William Verplank (levels of automation); Parasuraman, Sheridan and Wickens (2000 model), 1978; 2000

Level

201 · Tool

Fits

Small and mid-size, Scale-up, Enterprise

Time to apply

An afternoon to map which of an AI system's decisions need a checkpoint; ongoing to run the review itself

What you need

a list of every decision the AI system currently makes or recommends on its own · a named person with the authority and the time to say no to an AI output · a way to log what a reviewer changed, rejected or approved

Human-in-the-loop is a design pattern in which a person checks, approves or corrects an AI system’s output before it takes effect, rather than letting the system act on its own. The idea predates machine learning by decades. Thomas Sheridan and William Verplank described it in 1978 while studying how much a person needed to see and approve when operating an undersea robot arm by remote control, and laid out a ten-level scale running from full manual control to full autonomy. Raja Parasuraman, Sheridan and Christopher Wickens formalized that scale in 2000 for information systems generally, splitting automation into four stages (gathering information, analyzing it, deciding what to do and acting), each of which can sit at a different level of human involvement. AI teams picked up the phrase later, for the same reason the original researchers cared about it: a system that fails in a predictable place needs a person standing exactly there.

Where the checkpoint sits in the process

A human-in-the-loop checkpoint has three parts in a fixed order: the AI system produces an output, a person reviews it, and only after that review does the action happen.

The output can be a document read, a risk score, a drafted reply, anything the system would otherwise send or act on by itself. The review can be as small as a single click of approval or as involved as redoing the analysis from scratch. What makes it a checkpoint rather than decoration is that the action waits on the person, who holds the power to say no. A widely used set of design guidelines for human-AI products, written by Microsoft researcher Saleema Amershi and colleagues, starts from the same point: make clear what the system can do, and support efficient correction when it is wrong.

Three positions a person can hold

Human-in-the-loop is one of three positions researchers use to describe how closely a person sits to an automated system, alongside human-on-the-loop and human-out-of-the-loop. Human Rights Watch and the Harvard Law School International Human Rights Clinic set out a widely cited version of the distinction in a 2012 report on autonomous weapons, a field that had to define these positions precisely because a mistake there costs a life. In their terms, a human-in-the-loop system can select a target or an option but cannot act without a person’s command. A human-on-the-loop system can act on its own, with a person watching who can override it. A human-out-of-the-loop system acts with no person in a position to intervene at all. Noel Sharkey traced the same staged pattern in the military robotics literature: systems move from a person authorizing every action toward a person merely supervising, level by level, rather than jumping straight to full autonomy.

Many systems described as human-in-the-loop work as on-the-loop in practice: the system already sent the email or approved the transaction, and the person’s job is to catch a bad one afterward. A checkpoint that sits after the action has already happened protects against repeat mistakes. It does nothing for the one that already went out.

What the regulation requires

Two current regulations name human oversight directly, and neither treats it as optional. Article 14 of the EU AI Act requires that high-risk AI systems be built so a person assigned to oversee them can understand the system’s limits, notice when it behaves strangely, and decide, in a specific case, not to use it or to override, disregard or reverse its output. The article names the risk by its research term, requiring the overseer to stay aware of “the possible tendency of automatically relying or over-relying” on the system’s output, what the research literature calls automation bias. The GDPR’s older Article 22 works from the other direction: it gives a person the right to demand human review of a fully automated decision that has a legal or similarly significant effect on them, plus the right to contest it. Neither text specifies what a checkpoint should look like day to day. NIST’s AI Risk Management Framework fills part of that gap, asking organizations to define who is responsible for AI oversight, document how that oversight runs, and keep a working way to disengage or supersede a system when the checkpoint fails. The OECD’s AI Principles, adopted by dozens of governments, compress the same expectation into one line: AI actors should build in “capacity for human agency and oversight” suited to the risk. This section summarizes what the texts say. It is not legal advice for a specific system; check the current regulation and a qualified advisor before relying on it.

Why the checkpoint stops working

A checkpoint fails quietly, turning into a formality nobody performs rather than breaking outright. Parasuraman and Dietrich Manzey reviewed decades of automation research in 2010 and found the same pattern across pilots, radiologists and machine operators: when a person is asked to monitor a system that is usually right, attention to the exceptions drops, even among trained experts who already know the risk exists. Linda Skitka and colleagues had already shown the mechanism directly in a flight-simulation study: people given a reliable but imperfect automated aid missed errors that people working without any aid caught on their own, because checking took more effort than trusting. Ben Green and Yiling Chen found the same weakness in a 2019 experiment on pretrial release and lending decisions: people shown an algorithm’s risk score could not judge how accurate it (or they) were, and did not adjust how much they relied on it to how well it performed. A checkpoint built without friction gets skipped. One the reviewer cannot explain gets rubber-stamped, a term MIT Sloan Management Review uses for oversight reduced to a click with no scrutiny behind it.

Which decisions need a checkpoint

Put a person in the loop where a wrong output is expensive to undo, where a regulator will ask who approved it, or where the customer affected will notice. Three questions sort most decisions. Is it reversible? A miscategorized product photo costs nothing to fix later; a wire transfer sent to the wrong account might not come back. Is it regulated? A lender using AI to score an application still has to give the applicant a specific, accurate reason for a denial rather than point at an unexplainable model, under the US Consumer Financial Protection Bureau’s reading of existing lending law. Does a customer feel it directly? An internal report an AI drafts for the team gets less scrutiny than a message it sends straight to a client.

Once a decision clears that bar, the next choice is how much of it to review. Full review, checking every output, fits small volumes or the highest-stakes cases. Sampling review, checking a statistically chosen slice and escalating anything unusual, is what most high-volume systems need instead, since reviewing thousands of daily outputs is the condition that produces rubber-stamping in the first place. Either way, write the escalation rule down before launch: which confidence score, category of output or customer segment routes automatically to a person, rather than leaving that judgment to whoever happens to be reviewing that day.

The last piece is measuring what the reviewer does with the checkpoint. An override rate near zero over months, on a system running at scale, is usually proof the review stopped happening rather than proof the AI reached perfection. Pushers builds this kind of standing checkpoint into every AI workflow it sets up under the AI for Real Work practice: a named owner, a defined trigger and a number someone watches.

How to apply Human-in-the-loop, step by step

1. List what the system decides on its own. Write down every output the AI system produces without a person touching it first: a message it sends, a score it assigns, a document it approves. A vague description like 'it handles onboarding' is not specific enough to check against. Result: a complete list of the system's decisions.
2. Sort each decision by stakes. Flag any decision that is hard to reverse, sits inside a regulated process, or reaches a customer directly. Those get a checkpoint. A low-stakes internal draft usually does not need one. Result: a short list of decisions that need a checkpoint.
3. Choose full review or sampling. Review every output for the highest-stakes decisions and the lowest volumes. For everything else, review a statistically chosen sample and route anything unusual to a person, since checking thousands of outputs a day is exactly the condition that turns review into a rubber stamp. Result: a review mode for each checkpoint.
4. Write the escalation rule before launch. Set the triggers that send an output to a person automatically (a confidence score below a threshold, a category, a customer segment) and write them down. Result: routing that does not depend on who is on shift.
5. Show the reviewer the reasoning. Show the person what the AI based its output on, not only the result. A reviewer who cannot see the reasoning approves by habit instead of by judgment. Result: a review screen with the evidence next to the answer.
6. Track the override rate and revisit it. Log how often the reviewer changes or rejects the AI's output. A rate near zero on meaningful volume usually means the review stopped happening rather than that the system reached perfection. Result: one number that shows whether the checkpoint still works.

Examples

A fintech's KYC document review

Illustrative: a payments company uses AI to pull a new business customer's registration details, ownership structure and ID documents into a structured file. A compliance analyst checks that file against the source documents before the account opens, not after, and signs off with their name attached. The AI does the reading. The analyst still makes the call a regulator will ask about.

A clinic's patient intake triage

Illustrative: a clinic uses an AI tool to sort incoming intake forms by urgency and draft a short summary for the clinician. The clinician reads the original form and the AI's summary side by side before deciding how soon to see the patient. The tool never decides on its own who gets seen first. It only prepares the case faster.

When to use it

Use it wherever an AI system's mistake would be expensive to undo, would need explaining to a regulator, or would land on a customer without warning. It fits decisions that combine financial or legal consequence with AI uncertainty: a lending decision, a medical triage step, a message sent straight to a client.

When not to use it

Skip a formal checkpoint on decisions that are cheap to reverse and invisible to anyone outside the team: a draft tag suggestion, an internal search ranking, a first-pass summary nobody acts on without further work. Adding a review step there mostly trains people to click approve without reading, the exact habit a working checkpoint exists to prevent.

Common mistakes

• Building a review screen that shows only the AI's answer, not what it based the answer on, so the reviewer has nothing to check the reasoning against.
• Reviewing every single output once volume is high enough that nobody reads them closely, which produces the same rubber-stamping a missing checkpoint would.
• Never measuring the override rate, so a checkpoint that stopped working keeps looking fine on paper.
• Giving the reviewer no authority to block the action, so the checkpoint is a courtesy notification rather than a gate.
• Applying the same review depth to every decision regardless of stakes, which wastes attention on the low-risk cases and starves the ones that need it most.

FAQ

What does human-in-the-loop mean?

Human-in-the-loop means a person checks, approves or corrects an AI system's output before it takes effect, instead of the system acting alone. The person sits between the AI's proposal and the action it would trigger, with the power to stop it.

What is the difference between human-in-the-loop and human-on-the-loop?

In human-in-the-loop, the system cannot act until a person approves that specific output. In human-on-the-loop, the system already acts on its own, and a person watches and can step in afterward. The second gives faster throughput and a weaker checkpoint.

Does the EU AI Act require human-in-the-loop?

It requires human oversight for high-risk AI systems under Article 14, including the ability to override, disregard or reverse an output and to stay aware of automation bias. The text does not mandate the in-the-loop design specifically; on-the-loop oversight can satisfy it if the person retains the power to intervene. This is a summary, not legal advice.

Is human-in-the-loop the same thing as human-in-the-loop machine learning?

No, and the shared name causes mix-ups. Human-in-the-loop machine learning means people label or correct training data to improve a model over time. The human-in-the-loop this page covers is a decision-time checkpoint: a person reviewing one specific output before it takes effect, not training the system.

How do you know if a human-in-the-loop checkpoint is working?

Track the override rate (how often the reviewer changes or rejects the AI's output) against volume. A rate that stays near zero as volume grows over months usually signals the review has become a formality. A working checkpoint shows some overrides and catches errors before they reach a customer or a regulator.

Sources

1. NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1
2. NIST AI Risk Management Framework, AI RMF Core (Govern, Map, Measure, Manage)
3. European Union, Regulation (EU) 2024/1689 (AI Act), Article 14, Human oversight, EUR-Lex
4. European Union, Regulation (EU) 2016/679 (GDPR), Article 22, Automated individual decision-making, EUR-Lex
5. Raja Parasuraman, Dietrich Manzey, Complacency and Bias in Human Use of Automation: An Attentional Integration, Human Factors 52(3), 2010
6. Linda J. Skitka, Kathleen L. Mosier, Mark Burdick, Does automation bias decision-making?, International Journal of Human-Computer Studies 51(5), 1999
7. Raja Parasuraman, Thomas B. Sheridan, Christopher D. Wickens, A Model for Types and Levels of Human Interaction with Automation, IEEE Transactions on Systems, Man, and Cybernetics, Part A, 2000
8. Thomas B. Sheridan, William L. Verplank, Human and Computer Control of Undersea Teleoperators, MIT Man-Machine Systems Laboratory, 1978, via MIT OpenCourseWare 16.422
9. Human Rights Watch and Harvard Law School International Human Rights Clinic, Losing Humanity: The Case against Killer Robots, 2012
10. Noel E. Sharkey, Saying 'No!' to Lethal Autonomous Targeting, Journal of Military Ethics 9(4), 2010
11. Ben Green, Yiling Chen, The Principles and Limits of Algorithm-in-the-Loop Decision Making, Proceedings of the ACM on Human-Computer Interaction 3(CSCW), 2019
12. Yunfeng Zhang, Q. Vera Liao, Rachel K. E. Bellamy, Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making, FAT* 2020
13. MIT Sloan Management Review, AI Explainability: How to Avoid Rubber-Stamping Recommendations
14. MIT Sloan Management Review, Justifying Human Involvement in the AI Decision-Making Loop
15. US Food and Drug Administration, Clinical Decision Support Software, guidance
16. FinCEN, Federal Reserve Board, FDIC, NCUA and OCC, Joint Statement on Innovative Efforts to Combat Money Laundering and Terrorist Financing, 2018
17. Consumer Financial Protection Bureau, Innovation spotlight: Providing adverse action notices when using AI/ML models
18. OECD, AI Principles, human-centred values and human agency and oversight
19. Partnership on AI, Human-AI Collaboration Framework & Case Studies
20. Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, et al., Guidelines for Human-AI Interaction, CHI 2019

Last updated Sep 25, 2026

Ilia PushinFounder, Pushers · Co-founder and COO, ARBI ExchangeIlia builds operating systems for growing companies in fintech and healthcare. Since 2021 he has run cross-border payments at ARBI Exchange, a licensed currency exchange in Thailand, including KYC and AML and the move into new jurisdictions.About the authorLinkedIn