We introduce LMCodec, a causal neural speech codec that provides high quality audio at very low bitrates. The backbone of the system is a causal convolutional codec that encodes audio into a hierarchy of coarse-to-fine tokens using residual vector quantization. LMCodec trains a Transformer language model to predict the fine tokens from the coarse ones in a generative fashion, allowing for the transmission of fewer codes. A second Transformer predicts the uncertainty of the next codes given the past transmitted codes, and is used to perform conditional entropy coding. A MUSHRA subjective test was conducted and shows that the quality is comparable to reference codecs at higher bitrates. Example audio is available at https://mjenrungrot.github.io/chrome-media-audio-papers/publications/lmcodec.

翻译：我们提出LMCodec，一种因果神经语音编解码器，能够在极低比特率下提供高质量音频。该系统的骨干网络是一个因果卷积编解码器，通过残差向量量化将音频编码为从粗到细的层级化令牌。LMCodec训练一个Transformer语言模型，以生成式方式从粗令牌预测细令牌，从而减少传输的编码数量。第二个Transformer根据已传输的过去编码预测下一个编码的不确定性，并用于执行条件熵编码。通过MUSHRA主观测试表明，其质量可媲美更高比特率的参考编解码器。示例音频参见：https://mjenrungrot.github.io/chrome-media-audio-papers/publications/lmcodec。