Speaker diarization

Speaker diarization answers the question "who spoke when?". Speech recognition turns what was said into text; diarization divides that text between speakers such as "Agent" and "Customer". In meeting recordings the number of speakers may not be known in advance; phone calls usually have two people, but that changes with three-way calls and transfers. Accurate attribution is essential both for a readable transcript and for any analysis built on it.

How it works

There are two approaches. With channel separation, each party is written to its own channel in the recording (stereo call recording); because each channel is transcribed separately, the speaker is known from the start. With a single-channel recording, a model splits the audio into short segments, extracts voice features from each and clusters similar segments into speakers. This second approach can make mistakes with similar-sounding voices, overlapping speech and short replies such as "yes" or "okay".

Why it matters

When speakers get mixed up, words end up attributed to the wrong person: an amount the agent quoted can look like something the customer asked for. Summaries, quality scoring, talk-time ratios and sentiment analysis built on the call transcript all depend on getting this right. That is why call centers prefer channel-separated recording wherever possible and fall back to single-channel recording only when there is no other option.