Agent Security › Human approval
Keep a human in the loop is incomplete advice. The useful question is where the human needs control. A practical consequence ladder for deciding which actions an agent can take alone.
"Keep a human in the loop."
It's one of the most common pieces of advice given to businesses using AI agents. It's also incomplete.
A human could technically be "in the loop" while an AI agent performs hundreds of actions without them seeing any of them. Or a person could be required to approve every tiny decision, removing much of the reason for using an agent in the first place.
The useful question isn't "should a human be involved?" It's "where does the human need to have control?"
For some actions, monitoring may be enough. For others, a person should approve the action before it happens. And for particularly consequential decisions, an AI agent may not be the right system to make the final decision at all.
Good human oversight isn't about putting a person everywhere. It's about putting them at the points where their judgement matters.
Human in the loop, often shortened to HITL, describes a system where a person participates somewhere in an AI process. That participation can happen in very different ways.
A person might review an AI recommendation, approve an action, correct a result, respond to an exception, monitor activity, investigate an alert, take over a task, or make the final decision.
These are all forms of human involvement. But they don't provide the same level of control. For AI agents, that distinction becomes particularly important because the AI may be able to act before a person sees the result.
With generative AI, the human often naturally sits between the AI and the outside world. You ask ChatGPT to draft an email. It produces the draft. You read it. You send it. The workflow already contains a human approval point.
Now turn that into an agent. The agent reads the incoming email, checks the CRM, looks up the order, writes the response and sends it. The human has disappeared from the execution path.
That may be exactly what you want. But it means the organisation has made a security and governance decision: this agent is authorised to communicate externally without prior human approval. That decision should be deliberate.
It helps to separate different forms of human involvement.
All three can be useful. The right approach depends on the consequence of the action.
If an agent needs approval before every search, every document it opens, every CRM lookup, every calculation and every tool call, then you haven't really created an autonomous agent. You've created a very complicated assistant.
For low-consequence, reversible actions, autonomous operation can make sense: searching approved sources, summarising documents, classifying information, extracting data, checking status information, preparing internal reports. The important thing is that the agent remains within its authorised environment.
A useful way to decide where humans belong is to stop asking "should AI be allowed to do this?" and instead ask "what happens if this action is wrong?"
Level 1 - Low consequence. Search approved information, summarise a document, classify an enquiry, extract structured data, check an order, draft internal notes. If these go wrong, the impact is generally limited and reversible. Control: autonomous, with logging and normal monitoring.
Level 2 - Moderate consequence. Update selected CRM fields, move documents, schedule internal tasks, create internal reports, change workflow status, prepare customer responses. An incorrect action could create work or confusion but can usually be corrected. Control: autonomous and monitored, or approval for exceptions.
Level 3 - Significant consequence. Send external email, change important customer information, issue a small refund, publish low-risk content, change an operational workflow, contact a supplier. These affect people or systems outside the agent's immediate environment. Control: conditional approval. Routine, tightly constrained actions may be autonomous; unusual actions stop for review.
Level 4 - High consequence. Make substantial payments, publish sensitive information, deploy production code, delete important records, change access permissions, create credentials, make contractual commitments, send highly sensitive data externally. Control: human approval before execution. The agent can often still research, analyse, prepare and recommend, but a person controls the final action.
Level 5 - Decisions about people. Employment decisions, disciplinary decisions, high-impact financial decisions, access to essential services, significant eligibility decisions. AI can potentially assist with analysis. But depending on the decision, applicable law and context, meaningful human decision-making may be required or appropriate. Control: AI assists, human decides. This is an area where organisations should also obtain appropriate legal or compliance advice for their jurisdiction and use case.
This isn't a universal legal framework. It's a practical way to start deciding where human judgement belongs.
One of the most useful patterns for AI agents is separating PREPARE from EXECUTE. An agent can do almost all of the work without necessarily having final authority.
This preserves much of the productivity benefit while keeping human control over consequential actions.
If a person receives 100 approvals every day, all formatted identically, with little useful context, they may start clicking approve, approve, approve. Now the organisation technically has a human in the loop. But the control isn't meaningful. This is sometimes called automation bias or approval fatigue.
A good approval request should help the person make a real decision. It should explain what the agent is trying to do, why, what system will change, what information is involved, what is unusual about this action, and what happens if it is approved. Human oversight only works when the person has enough information and time to exercise judgement.
Imagine an agent asks: "Approve tool execution 7FA92?" That's technically an approval. It's not meaningful oversight. Compare it with:
Finance Agent wants to pay £4,820 to Supplier X against invoice 4732. Supplier: previously approved. Invoice amount: matches purchase order. Bank details: changed three days ago. Reason for escalation: bank details changed. Approve or deny.
Now the human knows why their judgement is required. The goal isn't merely to insert a person. It's to give that person a useful decision.
Not every transaction needs equal scrutiny. Imagine a finance agent that processes hundreds of routine invoices. The business might decide that an approved supplier, with unchanged bank details, a matching purchase order and an amount below £500, may be processed automatically. But a new supplier, a changed bank account, an amount above £500, a duplicate invoice or an unexpected currency requires human approval.
This allows the agent to handle routine work while people concentrate on unusual or consequential situations. That's much closer to how useful autonomous systems should work.
Authority doesn't have to be binary. An agent might have: £0 to £100 autonomous; £101 to £1,000 manager approval; £1,001 to £10,000 finance approval; above £10,000 two-person approval.
The numbers will depend entirely on the organisation. The important idea is that authority can have levels. The same principle can apply to refunds, discounts, purchases, publishing, customer communication, data access or production changes.
There is one place where human involvement becomes particularly important: changing what the agent itself is allowed to do.
An agent shouldn't normally decide "I need greater access, so I'll give myself greater access." Changes to permissions, credentials, spending limits, approval thresholds, security controls, connected systems, agent creation or delegation authority should generally sit outside the agent's autonomous control. The worker should not issue its own security pass.
Human approval can also reduce the consequences of prompt injection. Imagine an agent reads a malicious webpage instructing it to send confidential information to an external address. If the agent can send messages autonomously, the malicious instruction may result in an external action. If external communication requires approval, the process stops. The human sees: "Agent requests permission to send customer data to an unknown address." The attack still reached the agent. But the action didn't automatically happen. This is why agent security needs layers.
We shouldn't make the opposite mistake either. Humans can miss things, misunderstand things, approve too quickly, be socially engineered, suffer alert fatigue or simply make mistakes.
Human approval should sit alongside permissions, technical boundaries, monitoring, authentication, logging, risk controls and incident response. Human in the loop is a control. It isn't the entire control system.
A person cannot meaningfully supervise an agent if they cannot see what it is doing. Useful oversight might include the current task, systems accessed, actions completed, actions blocked, approvals requested, money spent, external communications, unusual activity and security alerts. The person supervising the agent should not need to read thousands of tool calls. They need useful information.
An agent might escalate when an action exceeds its authority, a financial threshold is crossed, a new recipient appears, a new supplier is created, sensitive data is involved, a restricted system is requested, an unusual tool is required, the agent repeatedly hits a boundary, confidence is low, instructions conflict, a task falls outside its defined purpose, an irreversible action is requested, or another agent requests greater authority.
When the agent reaches the edge of its authority, don't make it guess. Escalate.
Another overlooked question: which human is allowed to approve the action? Not every person should be able to approve everything. Marketing Manager for publishing. Finance Manager for payment. IT Administrator for system access. Data Protection lead for a sensitive data exception. Business owner for a major authority change. An approval process is only useful if the approver actually has the authority to make the decision.
Imagine an agent needs approval for a payment. It shouldn't be able to search the organisation for someone willing to approve it. The approval route should be predefined: payment above £1,000 to the Finance Manager, production deployment to the Engineering lead, permission change to the system administrator. This prevents approval itself becoming another obstacle for the agent to route around.
For consequential actions, keep a record of the agent, the requested action, the reason, the time, the relevant context, the approver, the decision and the outcome. This helps answer an important question later: who authorised this? That's useful for security, governance and accountability.
Human oversight becomes particularly important when agents run continuously. A person may not be watching at 3am. That means an always-on agent needs predefined rules for what it can do when nobody is available. Routine monitoring continues. Low-risk internal action continues. External communication queues. A high-value payment waits. A security anomaly pauses. A critical incident revokes access and alerts. Autonomy should not silently increase just because the human is asleep.
Every approval workflow needs a timeout rule. If an agent asks for approval and nobody responds, does it wait, retry, escalate, cancel or continue anyway? For consequential actions, no response should not automatically mean approval. The default should be defined in advance.
People also need the ability to intervene after an agent has started working. Depending on the system, that may mean pause task, cancel action, revoke tool, remove permission, stop agent, stop all agents. Human control shouldn't exist only at approval points. There should also be a way to regain control when behaviour becomes unexpected.
This leads to a useful distinction. A business doesn't necessarily need a person involved in every action. But it should maintain meaningful human control over purpose (what the agent is supposed to do), authority (what it is allowed to do), boundaries (what it can never do), escalation (when it must ask), oversight (whether we can see what it is doing) and intervention (whether we can stop it). That's much more useful than simply saying keep a human in the loop.
For each agent, ask:
If you cannot clearly answer those questions, "we have a human in the loop" probably isn't enough.
We're building Agent Control around the idea that businesses need to decide where autonomy ends before agents start working. That includes defining agent authority (what it can do autonomously), approval gates (which actions need a person), approvers (who can approve them), escalations (what happens at a boundary), activity (what the agent actually did) and intervention (whether it can be paused).
The objective isn't to make a person approve everything. It's to make sure people retain control over the things that matter.
Agent Control is in build. Leave your email and we will let you know when it is ready.
As AI agents become more capable, we'll hear "keep a human in the loop" more and more often. But that phrase isn't enough. The real questions are: which human? At what point? With what information? With what authority? Before or after the action? And what happens when they're not there?
Good agent security doesn't require humans to watch everything an AI does. It requires something much more deliberate. Put humans where consequences begin. Let agents handle the work they're authorised to do. Monitor what matters. Escalate when they reach the edge of their authority. And keep consequential decisions under meaningful human control.