How to Test a Turkish AI Phone Assistant: 20-Point Checklist

The most reliable way to test a Turkish-speaking AI phone assistant is not to listen to a demo recording, but to run scenarios built from your own call reasons over a real phone line, with a mix of callers, and score the results against criteria you set in advance. In this article we turn those criteria into a 20-point checklist, a weighted scoring sheet and a two-week pilot plan. You can use the list whichever vendor you are evaluating.

If you are new to the topic, start with what an AI voice agent is; here we focus on testing.

Before you start: preparation

A good test starts with a good set of scenarios. Before the pilot, prepare the following:

• Real call reasons: Group recent calls by reason and pick the 10–15 most common. Write your tests around that list.
• Caller profiles: Bring in at least a few people of different ages, regions and speaking speeds. People who do not know the script make more realistic calls.
• Lines and settings: Call from a landline, a mobile, a car and a noisy place.
• Expected behaviour: Write down the "right outcome" for every scenario in advance: should the agent book the appointment, hand over, or give information and end the call?

The 20-point checklist

Try each item with several different callers and record the result in the scoring sheet below.

Understanding and reading back

1. Accents and dialects. Run the same scenario with callers from different regions and age groups. Does the agent understand the words, or does it keep asking certain speakers to repeat themselves?
2. Noisy environments. Try traffic, café chatter, speakerphone and in-car calls. Background noise should neither interrupt the agent nor turn into the wrong text.
3. Numbers and amounts. Say things like "iki yüz elli" (two hundred and fifty), "bin iki yüz elli lira" (1,250 lira) and "üç buçuk" (three and a half). Check that the agent both understands the amounts and reads them back correctly.
4. Dates and times. Check that expressions such as "haftaya salı" (Tuesday next week), "öbür gün öğleden sonra" (the afternoon after tomorrow) and "akşam yedi gibi" (around seven in the evening) become the right day and time, and that the agent repeats the date and asks for confirmation.
5. Turkish ID numbers and long digit strings. Say an eleven-digit national ID number, a phone number and an order number, grouped in different ways. The agent should read the string back to confirm it. Ask yourself whether you really need this information by voice; if you do, keypad entry (DTMF) may be safer.
6. Addresses and proper names. Test neighbourhood, street, building and complex names, personal names and brand names. Adding frequently used names to the knowledge base reduces errors.
7. Licence plates. Say plates such as "34 ABC 123" letter by letter and with the common Turkish spelling convention ("Adana'nın A'sı", A as in Adana). For businesses such as car service centres this item becomes critical.

Conversation flow

8. Interruptions. Talk over the agent and correct a detail. It should stop and listen, but it should not halt every time someone says "hı hı" or "tamam" (OK) (barge-in).
9. Silence and pauses. Pause while giving a number, say "bir dakika, bakayım" (one moment, let me check) and stay silent for a few seconds. The agent should not cut you off too early, and after a long silence it should politely check you are still there. Endpointing governs this balance.
10. Response speed. Do replies arrive at the pace of a natural conversation, or does the agent wait long enough for callers to think the line has dropped? Judge this from the impressions of different testers rather than a stopwatch (latency).
11. Off-topic questions. Ask about the weather, a competitor or something unrelated to your business. The agent should steer politely back to the point rather than drift into small talk.

Limits and error handling

12. Handover to a person. Say "I'd like to speak to someone", play an angry caller, and make a request the agent cannot handle. The call should reach the right team with a summary of the conversation; if nobody is available, a callback request should be created. We cover how to design this in our warm transfer guide.
13. Authority limits. Ask for things the agent must not decide, such as medical advice, a discount or a refund approval. It should decline and route you to the right person.
14. Error handling. What happens when the calendar or a connected system does not respond, or when a caller has to explain the same thing a third time? The call must not drop silently; the agent should explain, record the request or hand over. Making up an answer to an unknown question also fails this item (hallucination).

Compliance and records

15. KVKK notice. If calls are recorded, callers must be informed before recording begins. Check the wording, its position in the flow and that it plays on every branch. See our KVKK guide for details. Telling callers up front that they are speaking with an AI assistant also builds trust; ask your legal adviser whether it is a legal obligation in your case.
16. Recording and transcript quality. Are names, numbers and dates correct in the transcripts? Is the summary clear enough for an operator to take over without listening to the whole call?

Operations

17. Number connection. Test with the setup your real business number is connected to, not a demo number. Do calls from different carriers reach the agent, and is the caller's number shown? We explain the options in our number connection guide.
18. After hours. Call after closing time, at the weekend and on a public holiday. Does the right greeting play, and are requests recorded? See our greeting and after-hours announcement templates for ready-made wording.
19. Concurrent calls. Have several people call at once. No call should hit a busy tone and quality should hold up (concurrent calls).
20. Reporting. At the end of the pilot, can you see from the reports which calls were resolved, which were handed over and why? Without reports you cannot evaluate the pilot.

Scoring sheet

Give each item 0 (fail), 1 (partial) or 2 (pass) and multiply by its weight. Items with a weight of 3 are critical: a solution that scores 0 on any of them should not go live, whatever its total. Adjust the weights to your sector; at a car service centre, for example, the licence plate item should be a 3.

With these weights the maximum score is 84. Do not treat the total as the only decision criterion: the status of the critical items and the kinds of errors you see in transcripts matter at least as much.

When an item scores low, separate the causes. A missing knowledge base entry, a vague handover rule or an over-sensitive interruption setting can be fixed in hours; those are configuration issues. A system that keeps misunderstanding the same accent or the same kind of digit string despite corrections points to a structural limit. List the two kinds separately in the pilot report and give structural problems the most weight when you decide.

Two-week pilot plan

During the real-caller phase, also note how your operators handle the calls that are handed over. The point of a pilot is to see not just the assistant, but how the assistant and your team work together.

Common testing mistakes

The same checklist can give very different results depending on how it is applied. The usual mistakes are:

• Only the person who set it up makes calls. Whoever wrote the scenarios unconsciously picks sentences the agent will understand. Always include people who do not know the script.
• Calling from a quiet room with a good headset. Your customers call from the street, the car and busy shops. Your test conditions should match.
• Testing only the happy path. The smooth booking flow matters, but the real differences show when callers change their mind, give wrong information and correct it, or hang up halfway.
• Scoring without reading transcripts. A call can look "successful" while the transcript holds a misspelled name or a misheard time. Score with the transcript in front of you.
• Not retesting after a fix. Changing the interruption setting can affect the noise item; changing instructions can affect handover. Retest the critical items after every change.
• One call per item. Do not count an item as passed after a single call; repeat the scenario with different callers.

Running a pilot with Speakline

We build Speakline in Turkey, Turkish-first, and you can use the checklist above to evaluate our product too. You can test your voice agent and listen to how it speaks before it goes live, and choose in the agent settings whether callers may interrupt it. You can port your existing number or connect the line you have with your carrier over SIP (BYOC), and limit the pilot to after-hours calls in the call flow.

Throughout the pilot, calls can be recorded and stored with Turkish transcripts; if a recorded step in the call flow has no KVKK notice before it, the designer warns you. Calls that are handed over reach the queue in the call center with a summary of the conversation, supervisors can monitor calls live, and results are available in reports. If you would like to plan a pilot around your own scenarios, get in touch.