This paper presents FunCodec, a fundamental neural speech codec toolkit, which is an extension of the open-source speech processing toolkit FunASR. FunCodec provides reproducible training recipes and inference scripts for the latest neural speech codec models, such as SoundStream and Encodec. Thanks to the unified design with FunASR, FunCodec can be easily integrated into downstream tasks, such as speech recognition. Along with FunCodec, pre-trained models are also provided, which can be used for academic or generalized purposes. Based on the toolkit, we further propose the frequency-domain codec models, FreqCodec, which can achieve comparable speech quality with much lower computation and parameter complexity. Experimental results show that, under the same compression ratio, FunCodec can achieve better reconstruction quality compared with other toolkits and released models. We also demonstrate that the pre-trained models are suitable for downstream tasks, including automatic speech recognition and personalized text-to-speech synthesis. This toolkit is publicly available at https://github.com/alibaba-damo-academy/FunCodec.

翻译：本文介绍了FunCodec——一个基础性的神经语音编解码工具包，它是对开源语音处理工具包FunASR的扩展。FunCodec为最新的神经语音编解码模型（如SoundStream和Encodec）提供了可复现的训练方案和推理脚本。得益于与FunASR的统一设计，FunCodec能够轻松集成到语音识别等下游任务中。随FunCodec一同发布的还有预训练模型，这些模型可用于学术或通用目的。基于该工具包，我们进一步提出了频域编解码模型FreqCodec，该模型能以更低的计算量和参数复杂度实现可比的语音质量。实验结果表明，在相同压缩比下，FunCodec相比其他工具包及已发布模型可取得更好的重建质量。我们还证明，预训练模型适用于包括自动语音识别和个性化文本转语音合成在内的下游任务。该工具包已在https://github.com/alibaba-damo-academy/FunCodec公开提供。