Turn on live captions during a quiet meeting with one clear speaker and you will think the problem is solved. Turn them on during a conference call with three people, a bad mic, and someone eating, and you will remember it is not. Speech-to-text has made a genuine leap in the last two years, and it has also hit a wall that no amount of marketing copy can talk past. Here is where the line actually sits, based on what these systems do outside the demo.
What works, and it is a lot
Clean audio in a common language is close to done. Whisper Large v3, the model most of the field is measured against, lands around a 2.7 percent word error rate on the clean LibriSpeech benchmark. In human terms that is roughly one wrong word in forty, which is better than most people type. If you are transcribing a well-recorded podcast, a single presenter with a decent microphone, or dictation in a quiet room, modern models will not be your problem.
Multilingual coverage is the other real win. Whisper handles ninety-nine languages, and while quality varies, the major ones are genuinely usable. A few years ago, transcription outside English was a research project. Now it is a checkbox. Batch processing, where the system has the whole recording and can take its time, is also strong. Deepgram's Nova-3 reports around 5.26 percent word error on batch audio, and for the common job of turning a recorded call into searchable text overnight, that is more than good enough.
Where it still falls apart
Now the wall. The headline numbers come from clean audio, and the real world is filthy.
One contact-center study told the whole story with three numbers from a single system: 92 percent accuracy on clean headset audio, dropping to 78 percent in a conference room, and collapsing to 65 percent on mobile calls with background noise. Same model, same language, same day. The only thing that changed was the room, and the room is exactly what you do not control. Background noise, crosstalk, and cheap microphones are not edge cases. For most real deployments they are the median case.
Accents remain a stubborn source of error, and it is a fairness problem as much as a technical one. Models trained mostly on standard American and British English degrade on strong regional accents and on second-language speakers, which means the people most likely to get a garbled transcript are often the ones already underserved by technology. This is getting better. It is not fixed.
Real-time is the other hard frontier. A model that scores beautifully in batch mode has to give up accuracy to run live, because it can no longer wait for the end of a sentence to decide what the beginning meant. Deepgram's own numbers show the gap, roughly 6.84 percent word error streaming against 5.26 percent batch. That spread is the price of being live, and for voice agents it is the difference between smooth and infuriating.
And then there is punctuation and speaker labeling, the unglamorous last mile. Getting the words right is one thing. Knowing where the sentence ends, who said it, and where the question mark goes is another, and it is where a technically accurate transcript still reads like a wall of mush.
What to actually pick
Match the tool to the job rather than the leaderboard.
- Offline transcription of recorded files, any budget: Whisper Large v3 is the sensible default, and if speed matters more than the last fraction of accuracy, Distil-Whisper runs about six times faster and stays within a point of it.
- Real-time voice agents and live captions: a streaming-first system like Deepgram Nova-3 will serve you better than forcing a batch model to go live.
- Noisy or multi-speaker audio: budget for cleanup and diarization as separate steps, and test on your actual recordings, not the vendor's samples.
The mistake I see most is trusting the benchmark number and skipping the test on your own audio. That gap between 92 and 65 percent is not hiding in a spec sheet. It is hiding in your particular room, your particular microphones, your particular speakers. Record ten real samples from the environment you actually care about, run them through two or three systems, and read the transcripts yourself. The winner is rarely the one with the prettiest published number, and you will know within an afternoon.