Denoising diffusion probabilistic models (DDPMs) have shown promising performance for speech synthesis. However, a large number of iterative steps are required to achieve high sample quality, which restricts the inference speed. Maintaining sample quality while increasing sampling speed has become a challenging task. In this paper, we propose a "Co"nsistency "Mo"del-based "Speech" synthesis method, CoMoSpeech, which achieve speech synthesis through a single diffusion sampling step while achieving high audio quality. The consistency constraint is applied to distill a consistency model from a well-designed diffusion-based teacher model, which ultimately yields superior performances in the distilled CoMoSpeech. Our experiments show that by generating audio recordings by a single sampling step, the CoMoSpeech achieves an inference speed more than 150 times faster than real-time on a single NVIDIA A100 GPU, which is comparable to FastSpeech2, making diffusion-sampling based speech synthesis truly practical. Meanwhile, objective and subjective evaluations on text-to-speech and singing voice synthesis show that the proposed teacher models yield the best audio quality, and the one-step sampling based CoMoSpeech achieves the best inference speed with better or comparable audio quality to other conventional multi-step diffusion model baselines. Audio samples are available at https://comospeech.github.io/.

翻译：去噪扩散概率模型在语音合成中展现出良好的性能，但实现高样本质量需要大量迭代步骤，这限制了推理速度。如何在提升采样速度的同时保持样本质量成为一项挑战。本文提出一种基于一致性模型的语音合成方法——CoMoSpeech，通过单步扩散采样实现语音合成，同时保持高音频质量。我们应用一致性约束，从精心设计的基于扩散的教师模型中蒸馏出一致性模型，最终使蒸馏后的CoMoSpeech获得优越性能。实验表明，通过单步采样生成音频时，CoMoSpeech在单个NVIDIA A100 GPU上的推理速度比实时快150倍以上，与FastSpeech2相当，使基于扩散采样的语音合成真正具备实用性。同时，在文本到语音和歌唱合成上的客观与主观评估显示，所提出的教师模型取得了最佳音频质量，而基于单步采样的CoMoSpeech在推理速度上达到最优，其音频质量优于或等同于其他传统多步扩散模型基线。音频样本详见https://comospeech.github.io/。