Language models have been successfully used to model natural signals, such as images, speech, and music. A key component of these models is a high quality neural compression model that can compress high-dimensional natural signals into lower dimensional discrete tokens. To that end, we introduce a high-fidelity universal neural audio compression algorithm that achieves ~90x compression of 44.1 KHz audio into tokens at just 8kbps bandwidth. We achieve this by combining advances in high-fidelity audio generation with better vector quantization techniques from the image domain, along with improved adversarial and reconstruction losses. We compress all domains (speech, environment, music, etc.) with a single universal model, making it widely applicable to generative modeling of all audio. We compare with competing audio compression algorithms, and find our method outperforms them significantly. We provide thorough ablations for every design choice, as well as open-source code and trained model weights. We hope our work can lay the foundation for the next generation of high-fidelity audio modeling.

翻译：语言模型已成功用于对自然信号（如图像、语音和音乐）进行建模。这类模型的一个关键组成部分是高质量的神经压缩模型，它能够将高维自然信号压缩为低维离散令牌。为此，我们提出了一种高保真通用神经音频压缩算法，该算法在仅8kbps带宽下，将44.1 KHz音频压缩为令牌，实现了约90倍的压缩率。通过结合高保真音频生成领域的进展与图像领域更优的矢量量化技术，并改进对抗性损失和重建损失，我们实现了这一成果。我们使用单一通用模型压缩所有领域（语音、环境、音乐等），使其广泛适用于各类音频的生成式建模。与竞争性音频压缩算法相比，我们的方法性能显著更优。我们对每个设计选择均进行了详尽消融实验，并开源了代码和训练模型权重。希望我们的工作能为下一代高保真音频建模奠定基础。