Denoising Diffusion Probabilistic Models have shown extraordinary ability on various generative tasks. However, their slow inference speed renders them impractical in speech synthesis. This paper proposes a linear diffusion model (LinDiff) based on an ordinary differential equation to simultaneously reach fast inference and high sample quality. Firstly, we employ linear interpolation between the target and noise to design a diffusion sequence for training, while previously the diffusion path that links the noise and target is a curved segment. When decreasing the number of sampling steps (i.e., the number of line segments used to fit the path), the ease of fitting straight lines compared to curves allows us to generate higher quality samples from a random noise with fewer iterations. Secondly, to reduce computational complexity and achieve effective global modeling of noisy speech, LinDiff employs a patch-based processing approach that partitions the input signal into small patches. The patch-wise token leverages Transformer architecture for effective modeling of global information. Adversarial training is used to further improve the sample quality with decreased sampling steps. We test proposed method with speech synthesis conditioned on acoustic feature (Mel-spectrograms). Experimental results verify that our model can synthesize high-quality speech even with only one diffusion step. Both subjective and objective evaluations demonstrate that our model can synthesize speech of a quality comparable to that of autoregressive models with faster synthesis speed (3 diffusion steps).

翻译：去噪扩散概率模型在各类生成任务中展现了卓越能力，但其推理速度慢的缺陷限制了其在语音合成领域的实际应用。本文提出一种基于常微分方程的线性扩散模型（LinDiff），旨在同时实现快速推理与高质量样本生成。首先，我们采用目标与噪声之间的线性插值设计训练扩散序列，而此前连接噪声与目标的扩散路径为曲线段。当减少采样步数（即拟合路径所用线段数量）时，相较于曲线，直线更易拟合的特性使得我们能在更少迭代次数下从随机噪声生成更高质量样本。其次，为降低计算复杂度并实现带噪语音的有效全局建模，LinDiff采用基于分块的处理方法将输入信号分割为小块。基于分块的标记通过Transformer架构实现全局信息的有效建模。通过对抗训练进一步减少采样步数并提升样本质量。我们测试了基于声学特征（梅尔频谱图）条件控制的语音合成方法。实验结果表明，即使仅使用单步扩散，模型仍能合成高质量语音。主观与客观评估均证实，我们的模型能以更快合成速度（3步扩散）合成与自回归模型质量相当的语音。