How to evaluate an AI receptionist: a 20-point checklist

Why answering the first call matters so much

In home services, the first call is often the only call. Eight in ten Americans say they generally do not answer cellphone calls from unknown numbers, and only 19% say they usually do^[3]. If a homeowner reaches voicemail and you call back from an unfamiliar number, there is a good chance they never pick up and book whoever answered. Speed compounds the problem: in an audit of US companies' web leads, firms that responded within an hour were nearly seven times as likely to qualify the lead as those that waited longer^[4].

That is why so many owners are looking at AI receptionists. The question is not whether software can answer a phone. It is whether it captures the job correctly, handles the dangerous calls safely and stays inside rules you control. The checklist below turns that into twenty specific points you can test.

The 20-point checklist

Score each point pass or fail from your own test calls. Group A to C covers everyday quality, D is safety, E to G cover booking, limits and control.

1. A. Greets with your approved company name and greeting, not a generic script.
2. A. Says clearly that the caller is speaking with an AI system when asked, and in your greeting if you choose to disclose up front.
3. A. Offers a route to a person, or an honest callback promise, at any point the caller asks.
4. B. Captures the caller's name and reads back the callback number digit by digit.
5. B. Captures the full service address with ZIP code and confirms spelling of the street name.
6. B. Identifies the trade and service type: repair, maintenance, estimate, install or warranty.
7. B. Records the problem in the caller's own words, plus key details such as equipment age, brand and when it started.
8. B. Captures access notes: gate codes, pets, parking, best entrance, who will be home.
9. C. Checks the ZIP code against your service area and handles out-of-area callers politely without promising a visit.
10. C. Recognizes work you do not take on and says so instead of booking it.
11. D. Gives your approved safety instruction for gas smells, carbon monoxide alarms, electrical burning smells, sparking and active flooding, such as leave the home and call 911 or the gas utility.
12. D. Alerts your on-call path for urgent calls: primary technician, then backup, within the time you set.
13. D. When nobody on the on-call path responds, tells the caller the truth and gives a clear callback expectation.
14. D. Never tells a caller that a dangerous situation is safe, and never diagnoses.
15. E. Books routine visits only into slots that are actually available, respecting job length and technician skills, or records a booking request with the preferred window when no tested schedule is connected.
16. E. Confirms the booked window back to the caller and does not double-book.
17. F. Answers only from facts you approved, such as hours, service area and service-call policy, and says it does not know otherwise.
18. F. Does not quote prices, discounts or warranty terms you have not explicitly approved.
19. G. Produces a recording, transcript and accurate summary for every call, available to you the same day.
20. G. Lets you change instructions, hours and the on-call path yourself, quickly, and plays the recording notice your state requires.

How to run the test calls

1. 01

   Write ten scripts from real calls

   Pull ten calls from the last few months that represent your mix: a routine tune-up, an emergency, a price shopper, an out-of-area caller, an angry repeat customer, a caller who rambles and a caller on a bad connection.

2. 02

   Use different callers

   Ask staff, family or friends with different voices, accents and speaking speeds to make the calls from their own phones. One owner calling ten times is not a test.

3. 03

   Call at the hours it will actually work

   Test after hours and at your busiest times. If it will handle overflow, test while your lines are busy.

4. 04

   Score against the checklist

   Compare each summary with what the caller actually said. A wrong digit in a callback number is a fail, however polite the call sounded.

5. 05

   Fix and retest

   Change the instructions, then repeat the failed calls. A good vendor helps you fix problems in days, not weeks.

For a fuller library of scripts with pass criteria by trade, see our guide to testing AI phone agents with real call scenarios.

Emergency handling: the pass-or-fail section

Some calls are dangerous. The CDC reports that more than 400 Americans die each year from unintentional carbon monoxide poisoning not linked to fires, more than 100,000 visit an emergency department and more than 14,000 are hospitalized, with gas and oil furnaces among the sources^[5]. A receptionist, human or AI, must never treat a carbon monoxide alarm like a routine booking.

Test each safety scenario at least twice, using different words each time: my CO detector keeps beeping, the alarm says carbon monoxide, my wife feels dizzy and the furnace is running. Pass means the system gives your approved instruction immediately, then alerts your on-call path and records exactly what happened. Fail means any delay to collect booking details first, any reassurance, or any attempt to diagnose.

Legal points to check before going live

Recording laws differ by state. Federal law allows a call to be recorded when one party consents^[1], but some states, including Florida, require all parties to consent^[2]. If you record calls, make sure the system plays a recording notice that fits the strictest state your callers are likely to be in.

Outbound calls are a separate question. In February 2024 the FCC ruled that calls made with AI-generated voices count as artificial under the Telephone Consumer Protection Act^[6], so outbound calls that use them fall under the TCPA's consent rules for artificial and prerecorded voices. If a tool offers automated callbacks or reminder calls, ask how it handles consent. This is not legal advice; check with counsel for your situation.

Questions to ask every vendor

• Can I make my own test calls before signing, using my real greeting and rules?
• What happens on a call the system does not understand?
• How does the on-call alert work, how fast is it and what happens if nobody answers?
• Does it book into my schedule today, or does it record booking requests? Which scheduling systems are tested and live, and which are planned?
• Who can change instructions, and how quickly do changes take effect?
• Where are recordings and transcripts stored, for how long, and can I delete them?
• How do I get my data out if I leave?

Ask the same questions of every product, including Ava. Its job-management connections and customer text updates are listed as Planned, so test the booking path you would actually use on day one.

Example: scoring two products

Example: a fictional roofing and gutter company tests two AI receptionists with the same ten calls. Product one sounds more natural and scores well on greetings, but misreads two of ten callback numbers and tries to book a caller reporting water coming through a ceiling light fixture before giving any safety instruction. Product two is plainer, reads back every number, gives the approved instruction to turn off power to that area only if it is safe and call 911 if there is any sign of fire, then alerts the on-call lead. The owner picks product two. Sounding natural matters, but intake accuracy and safety decide whether the system protects the business.