Baseten leads Coval’s voice AI benchmark by sitting on the quality-latency Pareto frontier: its STT deployment is about 5× faster than OpenAI’s while also delivering the lowest WER. Production voice AI performance is not just about model choice, it depends on the full inference stack, from STT and TTS latency to orchestration, networking, autoscaling, and deployment placement.
Coval is a voice AI evaluation platform that also helps teams test, compare, and improve AI agents with reproducible, independent benchmarks. Its open benchmark methodology highlights the core inputs behind every leaderboard result: the dataset, provider model versions, normalization pipelines, and latency measurement approaches.
In Coval’s early-access benchmarks in early September 2026, Baseten is positioned at the top of the leaderboard, sitting on the quality-latency Pareto frontier. The benchmark measures what end-users experience: for speech-to-text (STT), that means evaluating both speed and quality together. For text-to-speech (TTS), latency (or time to first audio) determines whether the agent feels responsive. Word Error Rate (WER) measures transcription quality and determines whether the agent understands the user.
In these tests, Baseten’s STT deployment is approximately 5× faster than OpenAI’s while also showing the best WER. This result shows that optimized inference can improve responsiveness without forcing teams to sacrifice transcription quality.
On the frontier of quality and latency
Voice agents require many models working together, but this invokes a network tax that adds latency for the end-user. Even a few hundred milliseconds of delay can ruin realism in a live conversation.
Baseten defines the Pareto frontier with Qwen3 ASR 1.7B Streaming for STT with both latency and quality: no provider in the current benchmark is faster while also producing a lower WER*.
STT*as of Sept 8, 2026
For TTS, Baseten still sits close to the quality-latency frontier even though performance across providers is more tightly clustered.
TTS + WERFor voice AI, inference infrastructure defines product quality
Voice agents require many models working together, but this invokes a network tax that adds latency for the end-user. Even a few hundred milliseconds of delay can ruin realism in a live conversation.
A typical voice stack sends audio to one provider for speech-to-text, forwards text to another provider for reasoning, then sends the response to a third provider for text-to-speech. Latency compounds across the voice pipeline. Every hop between STT, LLM, TTS, orchestration, and networking adds latency the user can feel. The fastest voice experiences come from optimizing the inference stack and tightening the distance between every part of the voice stack.
The path to lower latency compounds: optimize each model, keep scaling behavior consistent, and co-locate the pipeline as close as possible to the end user.
Benchmarks should measure the inference layer
That’s why the right benchmark for a voice agent can’t just measure a single model’s throughput; it must include the full turn from when a user stops speaking to when the system starts speaking back. That means measuring how quickly speech-to-text streams or finalizes the transcript, how fast the LLM begins responding, how quickly text-to-speech produces first audio, and how much delay comes from voice activity detection (VAD), telephony, orchestration, networking, autoscaling, and concurrency.
Without that full-stack view, a strong median (p50) latency can still hide a p95 latency that hurts 5% of conversation turns for every user, and degrades the overall voice agent experience in production.
For teams building voice AI, the challenge goes beyond choosing the model(s); it’s deciding which combination of providers, models, infrastructure, and evaluation methods will make the product feel better in production.
Benchmarks are more complete with infrastructure
At Baseten, we’re not only optimizing for peak speed, but for the reliability and consistency production voice agents need. Reliability matters as much as customizability; for voice workloads, a fast median is not enough if latency swings under load. Dedicated hardware helps avoid noisy neighbors, which can keep the latency distribution tighter and performance more consistent. In Coval’s benchmark, that consistency is accounted for.
The “best” configuration depends on the use case:
Streaming STT needs low time to first transcript and stable finalization behavior.
Batch transcription can optimize more heavily for throughput and cost.
TTS needs low time to first audio for interactive agents, but may optimize differently for non-streaming generation.
Full voice agents need all components to fit inside one end-to-end conversational latency budget.
Coval’s benchmark currently runs from a fixed region by design, which is useful for fairness and reproducibility. But in production, workload placement is one of the most important levers voice builders have. The benchmark gives teams a baseline; deployment architecture determines how that baseline translates in production.
Baseten’s deployments slot into Coval’s benchmark as normal API endpoints, but with dedicated inference levers underneath. That means the benchmark can reflect more than model quality; it can show what happens when inference is tuned for production voice AI, including reliability and performance consistency.
Get started
Deploy the benchmark models in one click from our Model Library:
Deploy: https://www.baseten.co/library/qwen3-tts-12hz-base-streaming-1-7b/
Deploy: https://www.baseten.co/library/qwen3-asr-1-7b-streaming/
From there, teams can plug the model into their own voice pipeline, test it on their own data, and tune the deployment for their latency, quality, and cost targets. If you need something more custom, reach out to talk to our engineers.
