It is challenging to accelerate the training process while ensuring both high-quality generated voices and acceptable inference speed. In this paper, we propose a novel neural vocoder called InstructSing, which can converge much faster compared with other neural vocoders while maintaining good performance by integrating differentiable digital signal processing and adversarial training. It includes one generator and two discriminators. Specifically, the generator incorporates a harmonic-plus-noise (HN) module to produce 8kHz audio as an instructive signal. Subsequently, the HN module is connected with an extended WaveNet by an UNet-based module, which transforms the output of the HN module to a latent variable sequence containing essential periodic and aperiodic information. In addition to the latent sequence, the extended WaveNet also takes the mel-spectrogram as input to generate 48kHz high-fidelity singing voices. In terms of discriminators, we combine a multi-period discriminator, as originally proposed in HiFiGAN, with a multi-resolution multi-band STFT discriminator. Notably, InstructSing achieves comparable voice quality to other neural vocoders but with only one-tenth of the training steps on a 4 NVIDIA V100 GPU machine\footnote{{Demo page: \href{https://wavelandspeech.github.io/instructsing/}{\texttt{https://wavelandspeech.github.io/inst\\ructsing/}}}}. We plan to open-source our code and pretrained model once the paper get accepted.

翻译：在确保生成语音高质量且推理速度可接受的同时，加速训练过程是一项挑战。本文提出了一种名为InstructSing的新型神经声码器，它通过集成可微分数字信号处理和对抗训练，能够在保持良好性能的同时，相比其他神经声码器实现更快的收敛速度。该模型包含一个生成器和两个判别器。具体而言，生成器包含一个谐波加噪声模块，用于产生8kHz音频作为指导信号。随后，该HN模块通过一个基于UNet的模块与扩展的WaveNet连接，该模块将HN模块的输出转换为包含关键周期性和非周期性信息的潜变量序列。除了该潜变量序列，扩展的WaveNet还将梅尔频谱图作为输入，以生成48kHz的高保真歌声。在判别器方面，我们结合了最初在HiFiGAN中提出的多周期判别器和一个多分辨率多频带STFT判别器。值得注意的是，InstructSing在4块NVIDIA V100 GPU的机器上，仅用十分之一的训练步数就达到了与其他神经声码器相当的语音质量\footnote{{演示页面：\href{https://wavelandspeech.github.io/instructsing/}{\texttt{https://wavelandspeech.github.io/inst\\ructsing/}}}}。我们计划在论文被接受后开源我们的代码和预训练模型。