Benchmark Reveals TTFT Alone Misleads Voice AI Latency Choices

The benchmark covers every layer of the voice stack, including LLM, speech-to-text, text-to-speech, and speech-to-speech. According to LiveKit, a voice agent is a latency budget with a language model inside it. TTFT is the interval between sending an inference request and receiving the first token back, but for voice it is only one term in a sum. Because TTS models need a complete clause or sentence before producing audio, LiveKit calls the resulting metric time-to-first-sentence (TTFS) and argues that TTFS is what users actually feel.

LiveKit's voice agents overview breaks a single turn into STT at roughly 100–200ms, LLM at 300–500ms with streaming, TTS at 100–200ms, and network at 50–150ms over WebRTC, putting the practical end-to-end target at 700ms to 1.2s. Kwindla Hultman Kramer, co-creator of Pipecat, has advised targeting 800ms median voice-to-voice latency, with 1,500ms acceptable for a proof of concept. Daily's earlier work on the fastest voice bot supplies the human baseline: typical human response time in conversation is around 500ms, and pauses beyond 800ms feel unnatural.

Daily's February 2026 voice-agent LLM benchmark translates that into an LLM requirement: natural conversation needs voice-to-voice under 1,500ms, which works out to roughly 700ms of TTFT budget for a text-mode LLM inside a transcription-to-LLM-to-voice harness. That 700ms number is the bar to hold every provider against.

The benchmark also warns against misleading TTFT readings. Artificial Analysis changed its default workload in March 2026 to 10k input tokens from 1k, which raises both TTFT and output speed. Tests run from a virtual machine in Google Cloud's us-central1-a zone, so network latency is baked in. For reasoning models, TTFT counts the first reasoning token, not the first answer token. Daily measures from request send to first usable token off the API, and notes that TTFT varies substantially between benchmark runs.

According to the Artificial Analysis API providers leaderboard retrieved on August 30, 2026, with a 10k-token input workload, the lowest measured first-chunk latency was Baseten's gpt-oss-120b (high) at 0.23s with 266 tok/s output, followed by Baseten gpt-oss-120b (low) at 0.24s, DeepInfra's Nemotron 3 Ultra at 0.28s, Cohere's North Mini Code at 0.32s, and Command A+ at 0.40s. Meanwhile, silicon vendors optimized for throughput show a different profile: Cerebras' gpt-oss-120b (high) had 0.49s TTFT but 1,697 tok/s output, and Mercury 2 from Inception had 3.07s TTFT, illustrating the tradeoff.

The report concludes that TTFT is the right entry point but the wrong finish line for voice. Providers that win TTFT and lose token throughput will not feel fast, and teams should evaluate both TTFT and TTFS when choosing an inference API.