Text-to-speech (TTS) models have achieved remarkable naturalness in recent years, yet like most deep neural models, they have more parameters than necessary. Sparse TTS models can improve on dense models via pruning and extra retraining, or converge faster than dense models with some performance loss. Thus, we propose training TTS models using decaying sparsity, i.e. a high initial sparsity to accelerate training first, followed by a progressive rate reduction to obtain better eventual performance. This decremental approach differs from current methods of incrementing sparsity to a desired target, which costs significantly more time than dense training. We call our method SNIPER training: Single-shot Initialization Pruning Evolving-Rate training. Our experiments on FastSpeech2 show that we were able to obtain better losses in the first few training epochs with SNIPER, and that the final SNIPER-trained models outperformed constant-sparsity models and edged out dense models, with negligible difference in training time.

翻译：近年来，文本到语音（TTS）模型在自然度方面取得了显著成就，然而与大多数深度神经网络模型类似，其参数量往往超出必要。稀疏TTS模型可通过剪枝与额外再训练改进稠密模型，或以部分性能损失为代价实现比稠密模型更快的收敛速度。为此，我们提出采用衰减稀疏度训练TTS模型，即采用较高的初始稀疏度以加速前期训练，随后通过逐步降低稀疏率以获得更优的最终性能。这种递减式方法区别于当前将稀疏度递增至目标值的方案，后者所需时间显著超过稠密训练。我们将该方法称为SNIPER训练：单次初始化剪枝与演化速率训练。基于FastSpeech2的实验表明，采用SNIPER训练能在最初几个训练周期内获得更优的损失值，且最终训练完成的SNIPER模型性能优于恒定稀疏度模型，并小幅超越稠密模型，而训练时间差异可忽略不计。