How an AI answers a call
An AI voice agent, also sold as an AI receptionist or voicebot, is a computer program that answers phone calls and talks with the caller in ordinary speech. Speech recognition turns your words into text, a language model decides what to say or do, and speech synthesis speaks the reply. The business's phone system hands it the call, and it hands the call on to a person when it should. This subject moves fast: we last checked every fact on this page on 26 October 2026, and we check them again every three months.
Purple: your call and the station working on itKey
- Purple: your call and the station working on it
- Screens: what each station hears, writes and says
- Dashed: the phone network and the internet, far longer than drawn
An AI agent answers this practice's calls · waiting for a call
Press a button, or click a screen to see that station at work.Press a button, or tap a screen to see that station at work.
The steps describe the AI agent. Switch to see the touch-tone voice menu it replaces, common from the 1990s to the 2010s.
Anatomy of an AI voice agent: one call from your mobile through the phone system and the agent's three stations, and over to a person at reception. The switch shows the touch-tone voice menu it replaces. The drawing is not to scale.
-
Follow one call
Press Play to watch the call, or scroll down and the drawing follows you one step at a time. Try it, under the drawing, makes your own call.Press Play to watch the call, or Next to go one step at a time. Try it, under the drawing, makes your own call.
-
Step 1 of 6
The phone system hands your call to the AI
You call your dental practice to move an appointment. The call crosses the phone network to the practice's phone system, here in a data centre. The phone system puts it through as it would to a desk phone, but this extension is an AI agent: programs on servers, with no person behind them. Its first sentence tells you that you are talking to an AI.
Does it have to say that it is an AI?
In the European Union, yes, since 2 August 2026. The AI Act says a system that talks with people must be built so that they learn they are dealing with an AI, unless a reasonably well-informed person would see it anyway. They must learn it at the latest when the conversation starts.
On the phone you cannot see who answers, so the first sentence is the natural place. The rules section below has the detail.
Sources: European Commission AI Act Service Desk: Article 50, transparency obligations (2024); European Commission AI Act Service Desk: Article 113, entry into force and application (2024)
-
Step 2 of 6
Speech recognition writes down what you say
You say: "Can I move my appointment to Friday?" Your voice reaches the agent as it travels on any internet call, in slices of 20 milliseconds, 50 a second. The first station, speech recognition, turns the sound into text as it arrives. Its screen shows your words.
How does it know you have finished speaking?
Mostly by waiting for a short silence. VoiceXML, a standard for voice menus from 2004, already had a setting for it: how long the recogniser waits after your last word before it decides you are done.
Set that wait short and the agent jumps in while you pause for breath. Set it long and you sit in silence before every answer. Whatever it waits comes out of the short gap that people leave between turns.
Sources: IETF RFC 3551: the RTP profile for audio and video (2003); W3C: VoiceXML 2.0, the standard for voice menus (2004)
-
Step 3 of 6
The language model decides what to say
The text goes to the language model, the station with two racks. It is a program trained on a huge amount of text, and it writes its reply a word at a time. On its own it knows nothing about this practice. So it works from what the practice gives it: instructions, facts such as the opening hours, and tools. Here it finds your Tuesday visit in the customer records and checks the calendar over the internet. Friday at 10:00 is free.
Can it get things wrong?
Yes. A language model writes the words most likely to come next, and a fluent sentence can still be false. That is why a business gives it a narrow job and its own facts and tools, such as the calendar, rather than letting it answer from memory.
It is also why the handover to a person in step 6 matters.
-
Step 4 of 6
Speech synthesis speaks the reply
The reply goes to the third station, speech synthesis, which turns the text into a voice. It can start on the first words before the model has written the last ones. The sound goes back the way your call came, through the phone system to your mobile: "Friday at 10:00 is free. Shall I book it?"
Why does every station have to be so fast?
Because people notice silence. In a conversation the gap between a question and its answer is a fraction of a second, and a longer pause sounds like hesitation. The numbers are below, under How long a pause can last.
So the stations overlap. Speech recognition writes while you talk, the model starts its reply as soon as you stop, and speech synthesis speaks the first words while the rest is still being written.
-
Step 5 of 6
You cut in, and it stops talking
Halfway through the reply you say: "Sorry, can I speak to someone?" Speech recognition kept listening while the agent talked. It hears you, the agent stops in the middle of a word, and the model reads what you said. People cut in on the phone all the time. An agent that talked straight over you would sound like a recording.
Could a voice menu be interrupted too?
Often, yes. VoiceXML, the World Wide Web Consortium's standard for voice menus, calls it barge-in: you press a key or speak over a recorded prompt, and the prompt stops.
The menu's author could also switch it off, for a legal notice or an advert the caller had to hear to the end.
Sources: W3C: VoiceXML 2.0, the standard for voice menus (2004)
-
Step 6 of 6
It hands you to a person
You asked for a person, so the agent asks the phone system to transfer the call. The phone system rings Anna's desk phone at reception, she picks up, and the agent leaves the call. A well-built agent also hands over without being asked: when it cannot understand you, or when the question is not one it may answer.
How does the call move without you dialling again?
The agent sends a transfer request on the call it is already on. On internet calls the standard for this, published by the IETF in 2003, is the REFER message: one side of a call asks the other to call someone else instead.
Your mobile stays connected the whole time. Only the far end changes, from the agent to Anna's desk phone.
Sources: IETF RFC 3515: the SIP REFER method, for transfers (2003)
The voice menu and the AI agent, side by side
Both answer the same call on the same phone system. What changed is the machine behind it: a fixed tree of keys then, a model that reads your words now.
Voice menu
AI agent
What it understandsITU-T Q.23; W3C; Eurostat
Keys
Each key sends its own pair of tones, and some menus also knew a short list of set words
Your own words
Speech recognition turns them into text
Who decides the next stepW3C; European Commission AI Act Service Desk
A fixed tree
Each key leads to one branch the business set up in advance
A language model
It works out a reply from what you said, within the business's instructions
Recordings
Prompts recorded in advance, or read out by a speech synthesiser
A new sentence
Written for your call by the model, spoken by speech synthesis
Must it say it is a machine?European Commission AI Act Service Desk (1, 2)
No rule
The European Union's AI Act came in 2024
Yes
In the European Union since 2 August 2026, unless it is obvious
How long a pause can last
An AI agent has to hear you finish, decide and start speaking inside the gap people leave between turns, and the phone network takes its share of that time too. These three numbers set the size of the gap.
-
208 ms
Across ten languages, people began their answer about 208 milliseconds after the question ended, on average. Japanese speakers were fastest at 7 ms, Danish speakers slowest at 469 ms.
PNAS: universals and cultural variation in turn-taking in conversation (2009)
-
600 ms
In a study of 380 listeners, a silence of 600 milliseconds before a "yes" was enough to make the answer sound less willing.
Journal of the Acoustical Society of America: silent gaps after requests (2013)
-
150 ms
The International Telecommunication Union counts 150 milliseconds one way, mouth to ear, as the delay below which most calls are not held back. Your words make that trip to the agent, and the reply makes it back.
How it came to be
- 1952
A machine hears ten digits
Three engineers describe a circuit that recognises the digits zero to nine spoken over a telephone line, 97 to 99% of the time. It works for one speaker at a time, after it has been tuned to that voice.
Journal of the Acoustical Society of America: automatic recognition of spoken digits (1952)
- 1963
Keys a machine can hear
The Bell System starts selling push-button Touch-Tone phones in two Pennsylvania towns. Each key sends tones a machine can hear, which makes voice menus possible.
- 1992
Say it, don't press it
AT&T announces it will handle collect calls by speech recognition: callers say "collect" to a machine instead of an operator. It expects to cut 3,000 to 6,000 of its 18,000 operators.
Christian Science Monitor: AT&T and speech recognition (1992)
- 2004
A standard for voice menus
The World Wide Web Consortium publishes VoiceXML 2.0, a common language for voice menus. They can listen for keys and for spoken words, and play recorded or synthesised prompts that a caller may interrupt.
- 2017
The Transformer
Researchers present the Transformer, a network design built on attention, at the NeurIPS conference. It was made for translating between languages, and today's large language models are built on it.
- 2024
Tell people it is an AI
The European Union adopts the AI Act. From 2 August 2026, a system that talks with people must let them know they are dealing with an AI.
European Commission AI Act Service Desk: Article 50, transparency obligations (2024)
What the law asks of an AI on the phone
In the European Union, an AI that answers calls falls under the AI Act, and anything it records falls under the privacy rules that cover every phone call. Some countries add customer service laws on top. These are the duties that shape how an agent behaves on a call, in plain words. They are not legal advice.
-
It tells you it is an AI
Since 2 August 2026, the AI Act requires the provider of a system that talks with people to build it so that they are told they are dealing with an AI. The message must be clear and come at the latest at the start, unless a reasonably well-informed person would see it anyway. Breaking this duty can cost a company up to 15 million euros or 3% of its yearly worldwide turnover, whichever is higher.
-
Recording a call needs a reason, and you must be told
European law keeps calls confidential: no one but the people on the call may listen in or store it without their consent. It leaves room for recordings the law allows in lawful business practice, for example as proof of a sale. France's data protection authority, the CNIL, adds that callers must be told about the recording, and about their right to object to it before the call ends.
legislation.gov.uk; Commission nationale de l'informatique et des libertés
-
In Spain, a person whenever you ask
Spain's customer service law covers large companies and essential services. It bans automatic answering as the only way to reach them. A bot or voice menu must let you ask for a person at any moment, from the start of the call, and that person must say who they are. The law came into force on 28 December 2025. Companies have until 28 December 2026 to adapt their customer service to it.
Where Voxbi fits
Voxbi sells this as the Voxbi AI Receptionist. It answers on an extension in your Voxbi phone system, on the numbers you already have, and transfers calls to colleagues and queues in the same system. In Belgium and Luxembourg it comes only with a Voxbi Managed plan.
See it in Voxbi