Human-in-the-loop AI in customer support

In customer support, the person usually comes in at one of three points. They approve each reply the AI drafts before a customer sees it, they take over the conversations the AI can’t answer, or they read what it said afterwards and fix what it got wrong, so it answers better next time.

In machine learning the phrase also covers people labelling training data or rating a model’s answers so it learns from them. IBM’s explainer defines it broadly, as a system in which “a human actively participates in the operation, supervision or decision-making”.

For a founder, where the loop sits decides how much of your day the AI gives back. Approve every reply and you’re still answering every question, only faster. Deacon is an AI customer support agent that answers your customers in seconds from your own docs and leaves you only the questions that need you, and it has a free plan.

Four places to put the person

Each costs you a different amount of time, and each guards against a different mistake.

Most small teams want the last two together. The AI answers what your docs cover, hands you what they don’t, and you read back through the conversations to see what it should have known.

Human in the loop vs on the loop and out of the loop

The three-way split is best known from arguments about military robots. A 2012 Human Rights Watch report, written with Harvard Law School’s human rights clinic, sorted machines by how much a person controls what they do. The same split fits a support chat.

In everyday use, human in the loop covers the first two, as IBM’s definition does with “supervision”. The third is the one customers fear.

What customers say about AI and reaching a person

In Gartner’s survey of 5,728 customers in December 2023, the top concern about AI in customer service was that it would get harder to reach a person. These are the published figures we found, read on 23 September 2026.

Gartner’s advice is about the handover, not about dropping AI. A chatbot “must communicate to the customer that they will connect them to an agent in the event that the AI cannot provide a solution”, then become a chat with a person “that picks up where the chatbot left off”. That way, it says, customers can trust they’ll get to a solution in the AI channel.

How much of your time the loop takes

Put rough numbers on it before you choose where the person sits.

Say your site has 400 support conversations a month, at about two replies each, and the AI can’t answer 15% of them. Approving every reply at 45 seconds each takes 800 × 0.75, or 600 minutes. You still write the 60 answers it couldn’t give, at 6 minutes each, another 360. That’s 16 hours a month, and every customer waits for you. Taking only the handovers is 60 × 6, or 360 minutes, which is 6 hours a month, while the other 340 conversations are answered in seconds, 3am included.

The share handed to you is your escalation rate, and it falls as you fill the gaps in your docs. The share the AI finishes alone is its containment rate.

What the EU AI Act asks of a support chatbot

The EU AI Act’s human oversight rule, Article 14, is for high-risk systems, which must be built so that people can oversee them, override their output and stop them. The high-risk uses listed in Annex III include recruitment, credit scoring and evaluating students’ learning outcomes, and customer service chatbots aren’t among them.

What does apply to a support chatbot is Article 50, which has applied since 2 August 2026. People must be told they’re interacting with an AI system from the start of the first interaction, unless that’s obvious. Deacon’s default greeting introduces it as your AI assistant, and the box visitors type in reads Ask AI on every plan. If you write your own greeting, keep it saying so. For what the Act asks of your business, ask a lawyer.

How Deacon keeps you in the loop

Deacon answers on its own, in seconds, so your customers never wait for you to approve a reply. It keeps to your content instead. It answers from what your docs say, and when they don’t cover a question it says it doesn’t know instead of guessing. It always declines questions about the visitor’s own account, order or data, and it takes no actions in other systems, so there’s no refund or deleted account waiting for your yes. Why support bots make things up explains why that matters.

You’re in the loop where it counts. When a visitor leaves their email or asks for a person, the owners on your team get an email. Reply from Deacon’s dashboard, and if the visitor still has the chat open, your words appear there under your first name. Deacon stops answering in that conversation until you press Hand back to Deacon, and until you reply, it keeps answering their other questions. Human handoff follows one conversation minute by minute.

You’re on the loop for everything else. Every conversation is saved and searchable by what visitors asked, and a filter picks out the ones tagged Unanswered or Wants a person. Instructions let you tell Deacon in plain words how to behave, and the Playground lets you try it before a customer does.

And the loop closes, which is the part most bots leave out. Every question your docs didn’t cover has an Answer this button. What you write becomes part of what Deacon knows, so it can answer the next person who asks, and Save & check asks Deacon the question again and shows you the reply it now gives. No model is retrained. Your answer becomes one more source Deacon answers from, and the more you answer, the less it needs you.

Your conversations also show you what customers need. Topics group similar questions, count the people who asked and label most groups, for example as a how-to, a bug report or a feature ask, so you see what customers are stuck on and what they want you to build next. You can try all of it on the free plan, which needs no card.