Choosing TTS Based on Sound Quality Was Too Slow for Conversations — Separating 'Design' and 'Production' with a Measured 2.5x RTF Difference

I tried to implement a diffusion-based TTS that generates voices from captions for conversational use, but it was 2.5 times slower than a pre-trained model on the same GPU. Even reducing the number of steps didn’t close the gap, so I settled on a two-tier approach: 'Voice design with diffusion, speech with pre-trained models.'