When should an AI agent stop and ask a person?
The short answer
Before any action that is hard to undo or that commits the company: sending, paying, deleting, signing, or changing a record that others rely on. Below that line, the agent can work alone and report what it did. Deciding where that line sits is the leader's job, and it should be written down before the agent starts.
Human in the loop
The industry term for this is “human in the loop”: a person takes part at chosen points in the agent’s work, usually to approve an action before it happens.
Anthropic describes agents that pause for human feedback at checkpoints, or when they run into something that blocks them. It also notes that agents commonly have stopping conditions, such as a maximum number of attempts, so that they stay under control.
OWASP, a well-known security community, lists “excessive agency” among the main risks of AI applications: an agent given more functions, permissions or independence than it needs. One of its recommended safeguards is to require a person to approve high-impact actions before they are taken.
Three kinds of action
A practical way to draw the line is to sort every action the agent can take into three groups:
- It may do these alone: read, search, summarise, prepare a draft.
- It may do these and then report: correct something it is sure of, and write down what it found and what it changed.
- It must ask first: send anything outside the company, pay, delete, sign, change a record that other people rely on.
Write the three lists down before the agent starts.
In my own work, my agents never send, never delete, and never change anything already in my mailbox or in our company system. They prepare, and I carry it out. The question Who checks the AI agent? describes how that works day to day.
The agent must also stop when it is stuck
Approval is one reason to stop. Being stuck is another. An agent that cannot find what it needs should say so and wait, not fill the gap with a guess. Write this into its instructions in plain words.
Do not ask for approval on everything
If every small step needs approval, people stop reading what they approve, and the approval becomes a click. Keep the “ask first” list short, and make every item on it a decision that really matters. That way, each request for approval gets real attention.