Ring the line about 40 times, with three or four callers reading from a written script: numbered dishes, extras, spice levels, postcodes inside and outside your delivery area, changes of mind, and a request for a person. Compare each kitchen ticket with what you ordered, and go live only when the safety-critical cases pass every time.
Vendor pages tend to lead with headline figures such as order accuracy or upsell rates; before you lean on one, find out how it was measured and on whose menu. Your menu has its own traps: number 14 against number 40, "medium" meaning different heat on different dishes, a delivery code just over the border. A script finds them before customers do. Configuration is a separate job, covered in setting up an AI phone line for takeaway orders; testing starts once the line answers calls and prints tickets.
Build the test sheet before you ring the line
Preparation takes an hour and makes every later call comparable. Gather five things: an export of your menu with dish numbers, names, prices and options; your spice scale and which dishes it applies to; your delivery list (the codes you serve, any paid zone, the minimum order and the fee); your opening and delivery cut-off times; and the sort of kitchen ticket you expect, printed or on screen. Then freeze the menu. A price change halfway through a test run makes half the results meaningless.
Recruit three or four callers with different voices and phones, and book a block of about three hours. Forty calls at three to five minutes each come to two to three and a half hours of talk time on their own, so have the callers ring at the same time where the line accepts simultaneous calls (worth testing anyway, because a Friday evening will), and log as you go. Check how your provider counts calls during a trial, because on usage-based plans test calls may be billed like real ones. That is still far cheaper than a month of wrong orders.
Log every call in one table, filling in the expected ticket before you dial. Here is a filled-in row (illustrative).
| Call | What the caller says | Expected ticket | Actual ticket | Result |
|---|---|---|---|---|
| 07 | "Number 14, and two number 22s, mild." (noisy line) | 1 x No. 14, 2 x No. 22 mild | 1 x No. 40, 2 x No. 22 mild | Fault, critical: wrong dish |
Add columns for the caller, the severity, the cause (menu data, a setting, speech recognition, something else) and the date the fix went in. The cause column is what turns a failed test into a to-do list.
Let AI draft the call scripts, then make them messier
Writing forty scripts by hand is dull, and a chat assistant is good at turning a menu into test calls. Paste the menu export and ask for scripts in a fixed format.
Here is a takeaway menu with numbered dishes, extras, a spice scale and a delivery list.
Write 12 phone-call test scripts a customer might say, each with the exact words,
the dishes and options that should appear on the kitchen ticket, and the rule being tested.
Include pairs of dish numbers that sound alike, orders with changes of mind,
spice requests in everyday words, and two delivery codes just outside the area.
Output a table: call, what the caller says, expected ticket, rule tested.
Expect scripts that are too neat. The usual output has polite full sentences, every dish number spoken clearly, and nobody interrupting. A typical entry reads "I would like number 14 and number 22, medium spice, please", with an expected ticket that is correct and a test that is far too easy. Rewrite each one the way people really talk: "Yeah, can I get a, um, fourteen, no wait, fourteen and two twenty-twos, medium, actually make one of them hot." Add background noise to some, a mumbled delivery code to others, and one caller who changes everything after hearing the total. Keep the AI's expected tickets, but check each against the menu by hand, because a wrong expectation makes a good line look faulty.
Numbered dishes: the calls that catch mishearing
Numbers are short, and short words are the easiest to mishear on a phone line. Pick the pairs on your menu that sound alike: 13 and 30, 14 and 40, 15 and 50, and so on up to 19 and 90. Call each pair at least twice, once on a clear line and once with noise; one call can carry two or three pairs, which keeps the group to about ten calls. Then add the cases below.
| Test | What the caller says | What should happen | Why it breaks |
|---|---|---|---|
| Single dish | "Number 12." | Reads back the number and the dish name | No read-back means nobody notices a wrong match |
| Sound-alike pair | "Fourteen." then "Forty." | Two different dishes, each confirmed by name | Teens and tens sound the same on a weak line |
| Quantity | "Two number 3s and a number 13." | 2 x No. 3, 1 x No. 13 | "Two number 3s" parsed as number 23 |
| Number and name clash | "Number 13, the beef noodles." | Asks which one is meant | The caller's number and name point to different dishes |
| Number not on the menu | "Number 99." | Says there is no such number | Guesses the nearest number |
| Spoken in digits | "One, three." | Treated as 13, confirmed | Read as 1 and 3, two dishes |
| Renumbered menu | The old number after you renumber | Uses the current menu only | Old numbers linger in the vendor's copy |
| Several at once | "Numbers 3, 5, 9, 12 and 14." | Five lines, each confirmed | The last item overwrites an earlier one |
A confirmation that names the dish is the cheapest safeguard on the whole line: "Number fourteen, beef chow mein, one portion. Is that right?" It turns a mishearing into a quick correction, so test that it happens every time. If it only reads numbers back, a caller who heard 40 and said 14 will not notice either.
Extras and changes: check the ticket, not the transcript
The transcript is what the AI thinks it heard. The ticket is what your kitchen will cook. They are not always the same, so for every call in this group compare the printed or on-screen ticket with your expectation, line by line, including the total.
| Test | What the caller says | Check on the ticket |
|---|---|---|
| Extra attached to the right dish | "Number 22 with extra sauce and number 3 as it comes." | Sauce on 22 only, at the right price |
| Add versus swap | "Rice instead of chips on the 12." | Chips removed, rice added, price difference correct |
| Removal on one portion only | "Two number 13s, no onions on one." | Two lines or one line with a clear note, never "no onions" on both |
| Set meal contents | "The meal for two." | Included items not also charged as extras |
| Size change | "Large on the 14." | Large price and portion label |
| Change of mind | "Make that three, not two." | One line with quantity 3, not two lines |
| Sold-out dish | An item you have marked unavailable | The assistant says so and offers an alternative |
| Total check | Any order of four or more lines | Total spoken equals the till's total |
How a ticket reaches the kitchen depends on your till or ordering system, and the same faults can come from the integration rather than the speech recognition. If your tickets arrive through an EPOS, the comparison of takeaway EPOS systems and their AI features shows what the receiving side can handle. Record the cause for every fault: "heard wrongly" and "heard correctly, written wrongly" need different fixes and different people to fix them.
Spice levels: map the words callers actually use
Menus say mild, medium, hot and extra hot. Callers say "not too hot", "as hot as you make it", "medium-ish" and "something for the kids". The assistant must turn the second list into the first without guessing, and the result must print where the cook will see it. First decide which dishes have a spice choice, what the default is when the caller says nothing, and whether any dish is always one level. Then test these phrases.
| Caller says | Pass | Typical fail |
|---|---|---|
| "Medium." | Medium, on the right dish | Medium applied to the whole order |
| "Not too hot." | Asks once: mild or medium? | Silently picks medium |
| "As hot as you do it." | Extra hot, confirmed once | Notes "hot" and loses the extra |
| "Mild for the kids." | Mild on the dish the caller names | Mild on every dish |
| "No chilli at all." | Recorded as no chilli, which differs from mild | Recorded as mild |
| "Two number 22s, one medium and one hot." | Two lines, each with its own level | Both medium, or the second lost |
| "Medium" on a dish with no spice option | Explains that the dish is not made spicy and offers a side | Writes "medium" in notes |
A realistic failure from the notes column: the assistant took "extra hot" correctly but stored it as free text at the bottom of the ticket, in small type, where a busy cook did not see it. The fix is to map spice to a modifier that prints beside the dish. Test it by reading the ticket from a metre away, the way a cook will. And keep spice separate from allergies in your test set. A caller who says "I can't have chilli" may mean a preference or a medical issue; the assistant should not decide which.
Postcodes and delivery area: inside, outside and on the boundary
A wrong "yes" sends a driver out of range or promises a time you cannot keep. A wrong "no" loses a sale. Both go in the log. Take your delivery list and choose eight test codes: three well inside, two on the edge, three well outside. Use real codes, because the line can only be tested against the list it actually checks, but pair them with made-up house numbers and names so no customer's address goes into the test log. Have the expected answer for each code ready before anyone dials.
| Test | Pass | Typical fail |
|---|---|---|
| Code inside the area | Accepts delivery and states the fee and the minimum order | Refuses, or quotes the wrong fee |
| Code well outside | Declines delivery and offers collection | Accepts the order |
| Code that shares its first characters with a delivered code | Checks the full code, or declines | Matches on the first half |
| Code read with sound-alike characters (B and D, M and N, zero and O) | Reads it back character by character and asks for confirmation | Accepts a misheard code silently |
| No code given, only a street | Asks for the code | Guesses the area |
| Inside the area, order under the minimum | States the minimum and offers to add items or collect | Accepts the order |
| Code in a paid zone | Quotes that zone's fee | Quotes the standard fee |
| Call after the delivery cut-off | Says delivery has stopped and offers collection or a later slot | Promises a time you cannot meet |
Read each test code in two ways, plainly and with a phonetic spelling, because real callers do both. On weak lines, letters such as B, D, E, P, T and V blur together, and so do M and N. The assistant should repeat the whole code back and wait for a yes. If it only reads back the street, a misheard code gets through unchecked. Also test collection: when delivery is declined, can the caller switch to collection without starting again?
Calls that leave the script: interruptions and hand-offs
Real calls are messy, and this group is where the line either copes or leaves a customer stuck. Ten situations cover most of it.
| Situation | Pass looks like | Fail looks like |
|---|---|---|
| Caller interrupts mid-sentence | Stops talking and listens | Talks over the caller and loses the order |
| Changes mind after hearing the total | Updates the order and gives a new total | Adds a second line instead of changing the first |
| Noisy background | Reads the order back and asks for confirmation | Accepts without a read-back |
| Long silence | Prompts once, then offers a person | Hangs up, or loops the same question |
| "Are you a robot?" | Says it is an automated assistant | Evades, or claims to be a person |
| "Can I talk to someone?" | Transfers the call, or takes a message if the shop is closed | Dead end, or rings a number nobody answers |
| Allergy question | Does not guess; reads only your approved allergen sheet or passes to staff | Answers from general knowledge |
| Call drops mid-order | No half-order reaches the kitchen | Sends a partial ticket |
| Same caller rings back | Notices a possible duplicate and checks | Creates a second order |
| Question about an earlier order | Passes to staff | Invents an answer |
Two of these deserve extra care. The allergy case is a safety matter, so a single wrong answer blocks launch. The "are you a robot" case is about honesty and may be a legal duty: if you have customers in the EU, the EU AI Act's transparency rule for systems that talk to people has applied since 2 August 2026, so ask your provider how the line identifies itself, and test that the greeting does it every time. For the wording of escalation rules, writing call scripts and escalation rules for an AI receptionist goes into detail.
Scoring a run: what blocks go-live and what only needs a fix
Decide the severity levels before the first call, so a bad result cannot be argued down afterwards.
- Critical (blocks launch): a wrong dish; an allergy answer given by guesswork; delivery accepted outside the area or refused inside it; no route to a person; an order lost silently.
- Major (fix before launch, or launch on a limited basis): wrong spice, extra or quantity; a total that differs from your till; delivery recorded as collection; a wrong phone number captured.
- Minor: clumsy phrasing, slow pauses, an extra question.
A pass rule I would start from: no critical faults across a full run, at least 98 per cent of ticket lines correct, every delivery decision right on your test codes, and each major fault either fixed or assigned to the vendor with a date. The numbers are yours to set; write them down before you begin. Here is an illustrative first run of 40 calls.
| Group | Calls | Clean | Faults | What went wrong |
|---|---|---|---|---|
| Numbered dishes | 10 | 9 | 1 | "Number 14" heard as 40 on the noisy-line call |
| Extras and changes | 8 | 6 | 2 | Extra sauce on the wrong dish; set-meal rice also charged as an extra |
| Spice levels | 6 | 4 | 2 | "Not too hot" defaulted to medium; "extra hot" went into notes |
| Delivery area | 8 | 7 | 1 | A code just outside the area accepted on a partial match |
| Off-script calls | 8 | 6 | 2 | Allergy question answered with a guess; after-hours transfer rang a dead number |
| Total | 40 | 32 | 8 | Four critical, four major |
Across those calls the tickets carried 124 lines, and 5 were wrong: 119 of 124 is 96.0 per cent. That sounds fine, and it is not. With an average of about three lines an order, a 96 per cent line accuracy means roughly one order in eight has a mistake somewhere, and in this run 5 of 40 orders did. Ask any vendor how its accuracy figure is measured: by line, by order or by call, on which calls, and in what conditions. Then measure yours the same way every time, so that runs are comparable. This run fails the pass rule on two counts, four critical faults and line accuracy below 98 per cent, so the line does not go live yet.
After a fix: re-test, soft launch and a weekly ten-call check
Sort the log by cause. In the run above, "number 14 as 40" is a speech problem and a read-back setting: switch on the dish-name confirmation. The extras faults are a menu-data problem: define each sauce as an extra attached to a parent dish. The out-of-area code is a rule problem: ask the provider to match the full code. The allergy answer needs a hard rule that routes the question to staff, and the dead transfer number is a typing error in a setting. None of these needs a new product, which is why the cause column matters.
After fixes, re-run the whole script, not only the failures, plus ten new calls. A second run of the same 40 plus ten new ones that comes back with 49 clean and one major fault (a spice level on a combo printing in notes) is a good place to start a soft launch: route after-hours calls to the line for two weeks while staff keep answering at peak times. Listen to the first hundred real calls on the recordings (where the line records calls and tells callers so), and compare tickets with what customers say they ordered. If something goes wrong with a live order, what to do when an AI receptionist gets a booking wrong covers the recovery routine.
Then keep the script alive. Re-run the full set after any menu change, price change, new delivery area, change of voice or model, or vendor update, and run a ten-call sample every week. Keep the log in one sheet with a date for each run, so you can show a vendor which cases changed, and so you can judge the line over a quarter using the method in measuring an AI receptionist's return in the first 90 days. An order line that passed in October can fail in November without anyone touching it, and the only way to know is to ring it again.
Questions about testing an AI order line
How many test calls are enough before going live?
Forty is a practical start: enough to cover each group of the script several times with different voices. Add about ten more after every fix. The aim is not a number but that the safety-critical cases, such as wrong dishes, allergy answers and delivery decisions, never fail across a full run, and that you can show the log to prove it.
Can I use an AI voice to make the test calls?
You can, for a clean and repeatable baseline run, but it will not mimic real callers: accents, interruptions, background noise and mumbling. Use the synthetic caller for the first pass, then people for the rest, including at least one person who never uses voice assistants and one who rings from a noisy room.
Should the AI line take card details over the phone?
Many set-ups avoid it by sending a payment link by text or taking payment on collection or delivery. If yours takes card numbers by voice, ask the provider how they are handled and stored, and check that nothing appears in transcripts or on tickets. Confirm the arrangement with your payment provider before you switch it on.
The vendor says a failed case is fixed. What now?
Re-run the whole script, not only the failed call. A fix to one rule can break another, and the log tells you which cases changed. If the same critical case fails twice after a fix, treat that as evidence that the product cannot handle your menu, and weigh whether to keep paying for it.
Further reads
- Square AI Voice Ordering for Restaurants: Set Up and Test (Beta) — Set up and test one specific voice ordering product.
- Pizza Phone Ordering System: What AI Costs per Month — What phone ordering costs per month for a pizza takeaway.
- Popmenu AI Answering: Does It Take Orders or Only Send Links? — Whether a phone assistant takes orders or only sends links.
- Can a Takeaway Take Orders on WhatsApp With AI? — Take orders on WhatsApp as a second channel.
- AI for Takeaways: What Is Worth Automating and What Is Not — Decide which takeaway jobs deserve automation at all.
- AI Ordering vs Delivery Apps: What a Takeaway Actually Pays — Compare own ordering against delivery app commission.
- Takeaway Delivery App Menus: Rewrite Descriptions and Photos With AI — A description formula, paste-in prompts, per-app photo checks, a modifier-group clean-up and a two-week test for takeaway delivery menus.
- AI Tools and AI Development: The Complete 2026 Guide — the AI hub, including every tutorial in the AI-for-business series.
Sources: the public product pages of several AI phone ordering vendors, read for what they claim about menus, delivery areas and accuracy; the EU AI Act's transparency duty for systems that interact with people.