In neural audio synthesis, neural vocoders and codecs are models that reconstruct waveforms from acoustic and latent representations, which are essential to the resulting audio quality. While current models are capable of generating perceptually natural speech, they still struggle with high-fidelity music and singing voice synthesis, as severe aliasing artifacts are introduced by non-linear activation functions and upsampling layers in existing architectures. Although various anti-aliasing techniques have been proposed in digital signal processing, their integration into neural vocoders and codecs remains under-explored. This paper incorporates differentiable anti-aliasing techniques into the activation and upsampling modules to bridge this gap, and thus presents Pupu-Vocoder and Pupu-Codec. We build a test signal benchmark to evaluate the anti-aliased modules, and validate our proposed models on speech, singing voice, music, and audio. Experimental results show that Pupu-Vocoder and Pupu-Codec outperform existing systems on singing voice, music, and audio, while achieving comparable performance on speech. Demos, codes, and checkpoints are available at VocodexElysium.github.io/AliasingFreeNeuralAudioSynthesis/.
翻译:在神经音频合成中,神经声码器和编解码器是从声学与隐层表示重建波形的模型,这对最终的音频质量至关重要。尽管现有模型能够生成感知自然的语音,但在高保真音乐和歌声合成方面仍面临挑战,因为现有架构中的非线性激活函数和上采样层会引入严重的混叠伪影。虽然数字信号处理领域已提出多种抗混叠技术,但其在神经声码器和编解码器中的集成仍探索不足。本文通过将可微分的抗混叠技术融入激活与上采样模块来弥补这一空白,进而提出了Pupu-Vocoder和Pupu-Codec。我们构建了测试信号基准来评估抗混叠模块,并在语音、歌声、音乐及通用音频上验证了所提模型。实验结果表明,Pupu-Vocoder和Pupu-Codec在歌声、音乐和音频方面优于现有系统,同时在语音任务上达到可比性能。演示、代码和检查点详见VocodexElysium.github.io/AliasingFreeNeuralAudioSynthesis/。