Recent advances in neural text-to-speech (TTS) models bring thousands of TTS applications into daily life, where models are deployed in cloud to provide services for customs. Among these models are diffusion probabilistic models (DPMs), which can be stably trained and are more parameter-efficient compared with other generative models. As transmitting data between customs and the cloud introduces high latency and the risk of exposing private data, deploying TTS models on edge devices is preferred. When implementing DPMs onto edge devices, there are two practical problems. First, current DPMs are not lightweight enough for resource-constrained devices. Second, DPMs require many denoising steps in inference, which increases latency. In this work, we present LightGrad, a lightweight DPM for TTS. LightGrad is equipped with a lightweight U-Net diffusion decoder and a training-free fast sampling technique, reducing both model parameters and inference latency. Streaming inference is also implemented in LightGrad to reduce latency further. Compared with Grad-TTS, LightGrad achieves 62.2% reduction in paramters, 65.7% reduction in latency, while preserving comparable speech quality on both Chinese Mandarin and English in 4 denoising steps.

翻译：近期神经文本到语音（TTS）模型的进展推动了数千种TTS应用步入日常生活，这类模型通常部署在云端为用户提供服务。其中扩散概率模型（DPMs）作为代表性方法，相较于其他生成模型具有训练稳定性高、参数效率优的特点。由于用户与云端之间的数据传输存在高延迟和隐私泄露风险，将TTS模型部署于边缘设备成为更优选择。在边缘设备上实现DPMs面临两个实际问题：首先，现有DPMs对资源受限设备而言仍不够轻量化；其次，DPMs在推理过程中需要大量去噪步骤，导致延迟增加。为此，本文提出LightGrad——一种轻量级TTS扩散概率模型。该模型配备轻量化U-Net扩散解码器与免训练快速采样技术，同时降低了模型参数量和推理延迟。此外，LightGrad还实现了流式推理以进一步降低延迟。与Grad-TTS相比，LightGrad在4步去噪条件下参数减少62.2%，延迟降低65.7%，同时在中英文语音合成中保持可比的语音质量。