This paper presents FunCodec, a fundamental neural speech codec toolkit, which is an extension of the open-source speech processing toolkit FunASR. FunCodec provides reproducible training recipes and inference scripts for the latest neural speech codec models, such as SoundStream and Encodec. Thanks to the unified design with FunASR, FunCodec can be easily integrated into downstream tasks, such as speech recognition. Along with FunCodec, pre-trained models are also provided, which can be used for academic or generalized purposes. Based on the toolkit, we further propose the frequency-domain codec models, FreqCodec, which can achieve comparable speech quality with much lower computation and parameter complexity. Experimental results show that, under the same compression ratio, FunCodec can achieve better reconstruction quality compared with other toolkits and released models. We also demonstrate that the pre-trained models are suitable for downstream tasks, including automatic speech recognition and personalized text-to-speech synthesis. This toolkit is publicly available at https://github.com/alibaba-damo-academy/FunCodec.

翻译：本文介绍了FunCodec，一个基础性的神经语音编解码工具包，它是开源语音处理工具包FunASR的扩展。FunCodec为最新的神经语音编解码模型（如SoundStream和Encodec）提供了可复现的训练配方和推理脚本。借助与FunASR的统一设计，FunCodec可以轻松集成到下游任务中，例如语音识别。与FunCodec一同提供的还有预训练模型，可用于学术或通用目的。基于该工具包，我们进一步提出了频域编解码模型FreqCodec，该模型能够以更低的计算量和参数复杂度实现可比的语音质量。实验结果表明，在相同压缩比下，与其它工具包和已发布模型相比，FunCodec能够实现更好的重建质量。我们还证明了预训练模型适用于下游任务，包括自动语音识别和个性化文本到语音合成。该工具包公开于https://github.com/alibaba-damo-academy/FunCodec。