Human-in-the-loop: designing AI agents that know when to ask

An AI agent can read a message, look something up, draft a reply and take an action. That last part is where most of the risk lives. A wrong answer in a draft costs a few seconds of review; a wrong refund, a wrong configuration change or a wrong message to a customer can cost far more.

So the question we ask first is not “how autonomous can this agent be?” but “which actions are safe to automate, and which need a person?”

Start with a job description

Every agent we build starts as a one-page job description:

• The job. “Answer order-status questions on WhatsApp.” Not “handle customer service”.
• The tools it may use. Read access to the order system. Nothing else.
• What it must never do. Promise refunds, change addresses, discuss pricing exceptions.
• When it hands over. Any request outside the job, any upset customer, any low-confidence answer.

If the job description is vague, the agent will be too.

Sort actions by how reversible they are

We put every possible action into one of three groups:

1. Read and inform. Looking up an order, summarising a document, classifying an email. Safe to automate fully, with logging.
2. Draft and propose. A reply, a ticket, a purchase request. The agent prepares it; a person approves it with one tap. This is where most of the time savings are.
3. Act on the world. Refunds, configuration changes, messages that commit the business. These stay behind explicit human approval until there is a long, reviewed track record.

Moving an action from group 2 to group 3 is a business decision, made on evidence, not a default.

Least privilege, literally

An agent should hold the minimum access its job needs. In practice that means separate read-only and write credentials, API users scoped to specific actions, and rate limits. If an agent is ever confused or manipulated by a cleverly worded message, the damage it can do is bounded by what it was allowed to touch.

Hand over with context

A hand-over that makes the customer repeat themselves is a failure. When an agent passes a case to a person it should include the conversation, what it looked up, what it was unsure about and what it suggests. The person should be able to resolve the case in one step.

Measure before and after go-live

Before an agent talks to real customers we collect a set of real past cases and agree what a good outcome looks like. The agent has to pass them. After launch, conversations are logged and sampled regularly, and every hand-over is a free lesson about where the job description needs to change.

The result

Agents designed this way are less dramatic than the demos, and far more useful. They take the repetitive volume off your team, they are honest about what they do not know, and they leave every decision that matters with a person who is accountable for it.