Approval as a control point, not a courtesy

Human-in-the-loop (HITL) is a design in which an automated system proposes an action and a person must approve, edit or reject it before it takes effect. The term comes from control systems and from the debate about autonomy in safety-critical machines, where the question has long been whether a person sits inside the decision loop, supervises it from outside or is absent from it.

AI agents bring that question into ordinary business software. An agent does not only produce text. It calls tools that change systems, so it pays a supplier, grants a permission, merges a change or emails a customer. Each of those is an action with an owner and a cost if it goes wrong. Human-in-the-loop is how an enterprise decides which of those actions the agent may take alone, and makes the others wait for the person accountable for them.

Three positions are worth keeping apart, because teams use them loosely.

  • Human in the loop. The action does not run until a person approves it.
  • Human on the loop. The action runs, and a person monitors the stream and can intervene, pause the agent or roll back.
  • Human out of the loop. The action runs with no person involved, and only the record is reviewed later, if at all.

A mature agent system uses all three, chosen per action according to risk, and moves actions between them as evidence builds up.

Why agents turn oversight into an engineering problem

A chat assistant needs no approval layer. It returns text, and a person decides what to do with it. The human is in the loop by construction.

An agent removes that default. Four properties make oversight something you have to design rather than assume.

  • It acts through tools. The agent’s output is a call to an API, a database or another system, so the effect is immediate and often external.
  • It is not deterministic. The same input can lead to different plans on different runs. You cannot test every path in advance the way you can a fixed workflow.
  • It reads untrusted input. Emails, tickets, documents and web pages can carry instructions meant to steer it, which is called prompt injection. A model that reads them can be talked into an action nobody intended.
  • Its reach compounds. An agent connected to many systems can chain actions across them faster than anyone could review after the fact.

Regulation points the same way. The EU AI Act requires high-risk AI systems to be designed for effective human oversight, including the ability to understand what the system is doing and to intervene or stop it. The OECD AI principles call for human agency and oversight. For an enterprise, that makes the approval layer part of the compliance case as well as the safety one.

Three oversight patterns

Approval is not one mechanism. These are the three patterns most enterprise deployments combine.

Pattern How it works Typical enterprise use What it costs
Approve before acting The agent proposes an action, execution pauses, and it resumes only when an authorized person approves Payments, refunds, privilege changes, production deploys, first contact with a key account Latency, and reviewer time on every instance
Review after acting The agent acts within policy, and a person reviews a sample or a digest on a schedule Ticket triage, CRM updates, routine replies from approved content, scheduling Errors live until review, so they must be cheap and reversible
Escalate on uncertainty The agent acts on cases that match policy with high confidence and routes the rest to a person with context Support queues, invoice exceptions, access requests, contract intake Needs a reliable signal for when the agent is unsure

The first is human-in-the-loop in the strict sense. The second is human-on-the-loop. The third is a hybrid, and in high-volume queues it usually carries the most value, because people spend their time only on the cases that need them. Maui’s Customer Support Manager works this way on a support inbox, answering from your own policies and passing hard cases on with the full history.

Deciding where approval belongs

The placement of approvals is a risk decision, and it should be made the way an enterprise makes any other control decision. Tier the agent’s actions by impact, not by how the agent feels about them.

Risk factor Lower risk Higher risk
Reversibility Can be undone in one step, like a tag or a draft Cannot be recalled, like a payment, an email or a deletion
Blast radius One record, one user Many records, many customers, a whole environment
Money No financial effect, or within a small threshold Moves funds or commits spend above a threshold
Data Internal, non-sensitive Personal data, regulated data, credentials
Audience Internal teammates Customers, regulators, the public
Privilege Uses existing access Grants, widens or removes access
Novelty An action it has taken many times under the same policy An action type or a context it has not seen

An action that is higher risk on any of these gets an approval before it runs. An action that is lower risk on all of them can move to review after acting. Write the result down as policy, per action type, so it can be reviewed and audited like any other control.

Two rules from established controls carry straight over. Separation of duties means the person who asked for an action should not be the one who approves it, so an agent working on a request from a manager routes approval to someone else. And approval authority should match the action, so a refund above a limit goes to finance and a production change goes to the system’s owner, not to whoever is nearest.

How approval works inside the system

For engineers, the core rule is simple. The approval must be enforced where the action is executed, not where it is planned.

  1. The agent proposes a structured action. Not “I will refund the customer”, but a typed object with the tool, its arguments, the reason and the evidence it relied on.
  2. A policy layer classifies it. Outside the model, code checks the action type, the arguments and the context against policy and decides whether it can run, needs approval or is refused.
  3. Execution pauses durably. The run is suspended in a way that survives restarts and can wait hours or days. Workflow engines like Temporal, or agent frameworks with an interrupt step like LangGraph, are common ways to do this.
  4. The request goes to the right approver. It is routed by policy to the accountable owner, in the tool they already use, like Slack, Microsoft Teams, email or an ITSM queue.
  5. The approver sees the exact action. The request shows the payload that will run, not a summary of it, along with the reason and the source data.
  6. On approval, preconditions are checked again. The world may have changed while the request waited. The balance may already be paid, or the user may have left. The action runs only if it is still valid.
  7. Execution is idempotent. Approving twice, or a retry after a timeout, must not run the action twice.
  8. Everything is recorded. The proposal, the policy decision, who approved, any edit they made and the result go into an audit log that cannot be altered.

An approval request that carries enough to decide on might look like this.

action        issue_refund
target        order 48213, customer acct 77120
amount        EUR 1,480.00
reason        duplicate charge, two captures 3 minutes apart
evidence      payment provider events, support ticket 99341
policy        refunds over EUR 500 need finance approval
approver      finance on-call, not the requesting agent's owner
expires       24 hours, then escalate to finance lead

Two failure modes are worth designing against. If the rule only lives in the prompt, a crafted email can talk the model past it, which is why the policy check sits in code. And if the approver only sees a friendly summary, they approve the summary, not the action, which is why the request shows the payload.

A worked example with IT access requests

Take a company where an agent handles access requests that arrive through the IT service desk.

  • Out of the loop. Closing duplicate tickets and attaching the requester’s org chart and current access. Easy to correct and touches no permissions.
  • Review after acting. Granting read-only access to a wiki space or a shared drive the requester’s team already uses. IT reviews a daily digest and can revoke in one step.
  • Escalate on uncertainty. Any request where the system, the role or the justification does not match a known pattern goes to the service desk with everything gathered so far.
  • Approve before acting. Admin rights, access to production data or to anything holding personal data. The request goes to the system owner, and production access also needs a second approver from security.
  • Act at once, then report. Revoking access when someone leaves. Speed is the control here, so the agent acts on the HR termination event and reports what it removed.

After a quarter, the record shows that wiki and drive grants were never reversed, and that the agent’s escalations for one finance tool were nearly always approved because the role mapping was missing. IT adds the mapping, and those requests move to review after acting. Production access stays behind two approvals, because the record cannot make that risk smaller.

Measuring whether approvals still work

Approvals decay. A queue that asks too often trains people to click yes without reading, and the control disappears while still looking present. Track a few numbers per action type.

  • Approved unchanged rate. A very high rate over many instances means the gate may be ready to move to review after acting.
  • Edit and rejection rate. Edits show where the agent’s policy or context is wrong. A rising rate is a signal to fix the agent, not to add reviewers.
  • Time to decision. Long waits mean the request is going to the wrong person or carries too little context to decide on.
  • Queue depth per approver. If one person approves more than they can read, the design is wrong.
  • Overrides and incidents. Any action that went wrong after approval is a review of the request itself. Ask what the approver could not see.

Move one action type at a time between patterns, keep the change in the policy history and keep reading the record afterwards.

The short version

Human-in-the-loop is not a switch on an agent product. It is a set of control points placed by risk, enforced outside the model, routed to the person accountable and measured so they do not decay into rubber stamps. Get the placement right and people spend their time on the decisions that deserve them. Get it wrong in either direction and the agent becomes either unsafe or pointless.