Decide first which enquiries AI will answer on its own and which it must hand over, then hire a person for the handovers: complaints, refunds, judgement calls and keeping the AI's answers accurate. Put that split in the job description, test candidates on real escalations and on correcting a wrong AI reply, and measure the person and the AI together.
That makes the job different from a traditional first customer service hire. The AI takes the repetitive questions, so the person's day is weighted towards the hardest conversations, plus a new duty: looking after the information the AI answers from. You're hiring for judgement, clear writing and calm under pressure, and you need to give them the authority to fix things, or every awkward case will land back on your desk.
The example throughout is a driving school with eleven instructors. Enquiries arrive through website chat, WhatsApp, email and the phone: around 600 a month about lesson availability, prices, test dates, rescheduling, and the occasional complaint. The owner had been answering most of them in the evenings, and had recently added an AI assistant to the website chat and WhatsApp.
Split the enquiries before you write the advert
List a month's enquiries by type, count them, and decide who handles each. The owner did this from two weeks of messages; the volumes below are illustrative. For more on which questions AI handles well, see AI customer service for small businesses: what to automate first.
| Enquiry type | Per month | Who handles it | Why |
|---|---|---|---|
| Prices, packages and what's included | 180 | AI alone | Answers come straight from the price list |
| Lesson availability and booking | 150 | AI, with booking link | The booking system shows real availability |
| Test dates and what to bring | 80 | AI alone | General guidance from written policy |
| Rescheduling and cancellations | 90 | AI drafts, person checks charges | Late-cancellation fees need a judgement call |
| Refunds on lesson blocks | 25 | Person only | Money and discretion |
| Complaints about lessons or instructors | 15 | Person only | Needs empathy and investigation |
| Anything the AI can't answer | 60 | Person | Handed over by the AI |
About 410 enquiries a month can be handled by the AI alone or with a booking link. The remaining 190 need a person to some degree. Those 190 conversations, plus the upkeep of the AI's knowledge, are the job. Define exactly when the AI hands over, using when an AI chatbot should hand over to a human as a guide: refunds, complaints, anything involving a charge, and any customer who asks for a person.
How many hours of human cover the AI leaves you
Now size the role. The driving school's sum, with illustrative handling times:
- Rescheduling checks: 90 × 3 minutes = 4.5 hours a month
- Refunds: 25 × 15 minutes = about 6 hours
- Complaints: 15 × 40 minutes (including calling the instructor) = 10 hours
- Other handovers: 60 × 8 minutes = 8 hours
- Phone calls the AI never sees: about 5 hours
- Knowledge-base upkeep and weekly checks of AI chats: about 3 hours a week, so 13 hours
That's roughly 47 hours a month, or 11 hours a week, before holidays, peaks and learning time. The owner advertised 16 hours a week across four weekday mornings and a Saturday, which gave room for growth and for the phone at busy times.
The AI's own running costs belong in the same sum. Many customer service tools now charge per AI resolution: Intercom's Fin is $0.99 per resolved outcome, and Help Scout, which is free for up to five users, charges $0.75 per resolution for AI Answers. At 410 AI-handled enquiries a month, that's about $406 or $308 respectively, if every one counted as resolved. The definition matters: Intercom counts an "assumed resolution" when a customer goes quiet for 24 hours after the AI's last answer, and Zendesk counts an automated resolution after two hours of messaging inactivity by default, so a customer who gave up may be billed as a success. On WhatsApp, Meta's own Business Agent has been charged per token since 1 August 2026, which Meta puts at roughly 4-5 cents a message, and from 1 October 2026 free-form service replies on the WhatsApp Business Platform become chargeable after the first 1,000 a month per business number. What customer service software with AI costs per agent covers the seat side of the bill.
A customer service job description with the AI's role written in
A customer service job description for this role should say plainly that an AI assistant handles routine questions, and what that means for the person. Here's the driving school's, filled in:
Customer Support Coordinator (16 hours a week)
About the role
We're a driving school with 11 instructors. An AI assistant answers
routine questions on our website chat and WhatsApp: prices, lesson
availability, test-day guidance. You'll handle everything it hands
over, and you'll keep the information it answers from accurate.
What you'll do
- Reply to handed-over enquiries within 2 working hours
- Handle rescheduling, late-cancellation charges and refunds up to
$150 without needing approval
- Investigate complaints: talk to the pupil and the instructor, then
agree a resolution with the owner for anything above $150
- Answer the phone during your hours
- Review a sample of the AI's conversations each week and fix wrong
or outdated answers in its knowledge base
- Tell the owner about patterns: repeated questions, new complaints,
answers the AI keeps getting wrong
What we're looking for
- Calm, clear writing and a friendly phone manner
- Good judgement about when to bend a rule and when not to
- Comfortable using a helpdesk and editing the AI's answers
(we'll train you on our tools)
Hours: Monday-Thursday mornings and Saturday, 16 hours in total.
Pay: [your rate] per hour.Two lines deserve attention. The refund limit gives the person real authority, so customers aren't told "I'll have to ask". And the weekly review of AI conversations makes quality of the AI part of the job, rather than something the owner forgets to do.
Where good candidates come from, and how to screen them
Customer service experience helps, but the closest match to this role is often someone who has handled upset people face to face: a receptionist at a busy clinic, a shift lead in a café, a hotel front-desk worker. They've learned to stay calm, find out what really happened and make a decision within their limits, which is exactly what the AI's handovers demand. Experience with a particular helpdesk tool is easy to teach; that temperament isn't.
The driving school added three short screening questions to its application form and read the answers before any CV:
- "Tell us about a time a customer was upset about something that wasn't your fault. What did you say?"
- "A customer says our website chat told them something that turns out to be wrong. What's your first message to them?"
- "What would you need from us to decide on a refund without asking the owner?"
Compare two illustrative answers to the second question. One applicant wrote: "I would explain that the chatbot is only an automated system and cannot always be accurate." Another wrote: "Sorry, our assistant got that wrong. Here's the correct information, and because we misled you, I'll make sure you're not out of pocket for it. I'll also fix the answer so it doesn't happen to anyone else." The first blames the tool; the second owns the problem for the business and thinks about the next customer. Only the second was invited to interview.
Interview exercises: fix the bot, calm the customer
Two short exercises tell you more than any question about experience. Run both in the interview, giving candidates 15 minutes for each.
Exercise 1: correct a wrong AI answer. Show them a real-looking chat where the AI got the policy wrong:
Pupil: I need to cancel tomorrow's 9am lesson, is that ok?
AI: No problem! Lessons can be cancelled free of charge up to
24 hours before the start time. I've noted your cancellation.
(Our actual policy: 48 hours' notice, otherwise the full lesson
fee is charged. The AI can't cancel lessons in the booking system.)Ask them to write the follow-up message to the pupil, and the corrected knowledge-base entry. A strong candidate apologises for the confusion, explains the real policy clearly, and uses judgement on the charge for this pupil, since the business's own assistant misled them. Their knowledge-base fix states the 48-hour rule and adds that the AI must hand over any cancellation within 48 hours, and must never say it has cancelled a lesson. A weak candidate either enforces the fee coldly or rewrites the entry without noticing the "I've noted your cancellation" problem.
Exercise 2: an escalation. Role-play a parent who has paid for a block of ten lessons and says the instructor arrived late twice and seemed distracted. They want a refund of the remaining six lessons. Good candidates listen, acknowledge, ask for dates, explain what happens next and when, and don't promise the full refund before investigating. Watch for anyone who either argues or gives everything away in the first minute.
Score both exercises out of 10. The driving school interviewed four people; the candidate hired scored 9 and 8, and was the only one who spotted that the AI had claimed to cancel a lesson it couldn't.
Authority, tools and access for week one
- Clear limits in writing. What they can refund, waive or offer without asking, and what comes to you.
- Access to the systems that answer questions. The booking system, the helpdesk or shared inbox, and edit rights on the AI's knowledge base.
- A disclosure line at the start of AI chats. If you serve customers in the EU, the AI Act's transparency duties, in force since 2 August 2026, require telling people they're talking to a chatbot. It's good practice anyway, and it sets expectations for the handover.
- A handover message that carries context. The person should see what the AI already told the customer, so nobody has to repeat themselves.
- A weekly sampling routine. Twenty AI conversations a week, read and scored, following a weekly sampling routine for AI support replies.
The handover itself deserves a fixed format, so the person never starts from nothing. Most helpdesk and chatbot tools let you set what the AI passes on. The driving school's version:
HANDOVER: [reason: refund / complaint / charge / asked for a person]
Customer: [name], pupil ref [number], instructor [name]
In their words: "[the customer's key sentence]"
What the AI already said: [one-line summary, incl. any policy quoted]
What they want: [refund / reschedule / call back / explanation]
Urgency: [lesson or test within 48 hours? yes/no]The "what the AI already said" line is the one that prevents the worst conversations, where a customer is told one thing by the bot and the opposite by a person.
A two-week start that sets the partnership up
The first fortnight decides whether the person trusts the AI enough to rely on it and doubts it enough to check it. The driving school's plan:
- Days 1-2: read 50 recent AI conversations and the full knowledge base, noting anything that looks wrong or unclear. Learn the booking system and the refund and cancellation policies with the owner.
- Days 3-5: handle handovers with the owner reading replies before they're sent. Take phone calls with the owner nearby.
- Week 2: handle handovers alone within the refund limit, with a short daily review of anything unusual. Make the first knowledge-base fixes from the list started on day one.
- End of week 2: a 30-minute review: what the AI gets wrong most, which rules are unclear, and what authority feels too tight or too loose.
The new coordinator's first knowledge-base fix shows what that upkeep looks like. Before, the entry the AI answered from read: "Cancellations: please give us plenty of notice if you can't make your lesson." That vagueness is why the AI had been inventing a 24-hour rule. After her edit it read: "Cancellations need 48 hours' notice; later cancellations are charged the full lesson fee. The assistant cannot cancel lessons. For any cancellation within 48 hours, or any request to waive the fee, hand over to the support coordinator." One precise paragraph removed a whole category of wrong answers.
The first 60 days: measuring the person and the AI together
Judge the person on the work that's theirs, and judge the AI and the person together on the customer's experience. The driving school tracked these, shown here with illustrative figures from month one and month two:
| Measure | Month 1 | Month 2 | What it tells you |
|---|---|---|---|
| Handovers answered within 2 working hours | 71% | 92% | The person's core responsiveness |
| AI answers found wrong in the weekly sample | 6 in 80 | 2 in 80 | Whether knowledge-base upkeep is working |
| Customers returning about the same issue within a week | 31 | 14 | Whether "resolved" really meant resolved |
| Complaints escalated to the owner | 9 | 3 | Whether the person is using their authority |
| Owner's evening hours on enquiries | About 8 a week | About 1 a week | The point of the whole exercise |
The second row is the telling one. As the new coordinator fixed outdated answers (an old price for the intensive course, a missing rule about test-day car hire), the AI's error rate fell, which in turn reduced the handovers she had to deal with. That's the partnership working. For the measures on the AI side in more depth, see how to stop an AI chatbot giving customers wrong answers.
The cost side, in this illustration, was straightforward. At $18 an hour, 16 hours a week came to about $1,250 a month (16 × $18 × 52 ÷ 12), plus the AI's resolution charges and the helpdesk. Set your own rate from local pay data. Against that, the owner got back about seven evening hours a week, and the returning-customer count halved. If you're weighing this against handing the whole thing to an outside firm, AI or outsourced customer service: which costs less runs that comparison.
Covering evenings and weekends without adding hours
A part-time person can't cover every hour, and the AI does its best work when they're not there, answering price and availability questions at 10pm. The gap is the handover that arrives on Saturday night. What the AI tells that customer decides whether Monday starts calmly.
The driving school's original out-of-hours handover message said: "Thanks, someone from our team will be in touch soon." Customers read "soon" as tonight, and several sent a second, crosser message by Sunday. The revised version said: "I can't sort this one myself, so I've passed it to our support coordinator, who works Monday to Thursday mornings and Saturdays. You'll hear back by 11am on Monday. If your lesson is before then, reply URGENT and the owner will see it." Repeat messages over the weekend fell away, and the owner received one or two genuinely urgent notes instead of every loose end. The coordinator starts Monday with a queue already sorted by urgency, which is the whole point of pairing a person with AI.
Hiring mistakes that make the AI look worse
These illustrative mistakes are common, and each one makes the AI seem less capable than it is.
- Hiring for speed, not judgement. A fast typist without the confidence to make decisions bounces every tricky case back to the owner, and the AI's handovers pile up.
- No authority. Without a refund limit, the person can only promise to ask, and customers who were already frustrated by a bot get a second delay.
- Making them babysit the bot. Asking the person to read every AI reply before it's sent defeats the purpose and wears them out. Sample instead, and fix the knowledge base.
- Nobody owns the knowledge base. Prices change, a new package launches, and the AI keeps quoting last season's figures. Make it someone's job, in the job description.
- Judging on volume. The AI takes the quick wins, so the person's closed-conversation count looks low next to a pre-AI benchmark. Measure the quality of the hard conversations instead.
Get these right and the first hire and the AI make each other better: the person teaches the AI through every correction, and the AI gives the person the time to do the part of the job that customers remember.
Hiring alongside AI: follow-up questions
Should my first customer service hire be full-time?
Size it from the enquiries the AI hands over, not from the total. Many small businesses find the handovers, complaints and knowledge-base upkeep fill a part-time role at first. Watch the handover queue for two months; if replies to handed-over customers start taking longer than your target, add hours.
Should the new person be allowed to change the chatbot's answers?
Yes, within agreed limits. They see the wrong answers first, so they should own the knowledge base the AI draws on: correcting outdated prices, adding missing policies, and flagging anything that needs the owner's decision. Changes to policy itself, such as cancellation terms, should still go through you.
What if the person sees the chatbot as a threat to their job?
Be clear from the advert onwards that the AI handles repetitive questions so the person can handle the conversations that need judgement. Measure them on the quality of handovers and complaint outcomes, not on volume the AI now takes, and involve them in improving the bot. People who shape a tool rarely resent it.
Can the AI and the person work in the same inbox?
Usually, and it's better when they do. Most helpdesk tools show the AI's reply history in the same conversation, so the person picks up with full context rather than asking the customer to repeat themselves. Check that handovers arrive with a clear flag and a summary of what the customer has already been told.
Further reads
- How to Measure Whether Your AI Chatbot Is Actually Working — The measures that show whether the AI half is working.
- How to Build the FAQ Your AI Chatbot Needs Before Launch — The knowledge base your new hire will look after.
- AI Chatbot Disclosure: What to Tell Customers at the Start of a Chat — What to tell customers at the start of a chat.
- AI Ticket Triage: Tag, Route, and Prioritise Support Requests — Route and prioritise what the AI hands over.
- How to Monitor AI That Talks to Customers: Hand-Offs and Errors — A monitoring routine for AI that talks to customers.
- WhatsApp Customer Service With AI: Setup, Costs, and Limits — If WhatsApp is where most of your enquiries arrive.
- AI or a New Hire? How to Decide Before You Recruit — Break a planned role into tasks, see which ones AI can absorb, cost hire against AI over a year, and set a clear trigger for recruiting anyway.
- How to Test Job Candidates' AI Skills in an Interview — An AI skills test for small-firm interviews: one realistic task with a planted error, a scoring rubric, and follow-up questions that reveal judgement.
- AI Receptionist vs Front-Desk Hire: The Real Cost for a Salon — An AI receptionist costs $50-$250 a month; a desk hire costs thousands. But the desk does more than phones. Here's how to compare them fairly.
- How to Set Up an AI-Assisted Hiring Process for a Small Team — A hiring pipeline for small teams where AI writes, summarises and schedules, people decide, and every rejection is read by a human first.
- Freshdesk vs Zendesk for a Small Support Team Using AI — Freshdesk bills AI per session and Zendesk per verified resolution. A costed three-person team shows which helpdesk fits, and what to ask sales.
- Customer Service QA Software With AI: What to Compare — The criteria that matter when choosing AI quality assurance software for a support team, five products compared, and a worked choice for six agents.
- How to Write a Job Advert for an AI-Savvy Admin Assistant — A bakery's admin advert built line by line: which AI skills to ask for, wording that attracts the wrong people, and a task that shows who checks their work.
- AI Tools and AI Development: The Complete 2026 Guide — the AI hub, including every tutorial in the AI-for-business series.
Sources: the series fact sheet for Intercom Fin, Help Scout and Zendesk resolution pricing and definitions, Meta Business Agent charges, WhatsApp Business Platform pricing from 1 October 2026, and EU AI Act Article 50.