We ran the same 1,000 utterances through 12 different STT/LLM/TTS stacks. Every combo got the same audio, the same prompt, the same hardware. Here's what we found.
TTFT, the only number that matters
Time-to-first-token is the gap between the user finishing speech and the agent starting to speak back. Under 800ms feels conversational. Over 1.2s feels robotic. Over 2s and customers hang up.
The winners
- Fastest: Deepgram Nova-3 + GPT-4o mini + Cartesia Sonic-3, 480ms median TTFT
- Best quality: Deepgram Nova-3 + Claude Sonnet 4.5 + ElevenLabs Multilingual v2, 1.1s TTFT, premium voice
- Cheapest viable: Deepgram Nova-3 + GPT-4o mini + Deepgram Aura, 420ms, $0.05/min COGS
If you're optimising for cost AND quality, GPT-4o mini is the sweet spot. Claude Haiku is 2x the price for marginal quality lift in most use cases.
What hurts latency
- Long system prompts (>2k tokens): +200-400ms
- Tool use that reaches an external API: +500ms to +3s
- Multilingual TTS with code-switching: +250ms
- Cold provider connections: +600ms first request
