Audio codecs power discrete music generative modelling, music streaming, and immersive media by shrinking PCM audio to bandwidth-friendly bitrates. Recent works have gravitated towards processing in the spectral domain; however, spectrogram domains typically struggle with phase modeling, which is naturally complex-valued. Most frequency-domain neural codecs either disregard phase information or encode it as two separate real-valued channels, limiting spatial fidelity. This entails the need to introduce adversarial discriminators at the expense of convergence speed and training stability to compensate for the inadequate representation power of the audio signal. In this work we introduce an end-to-end complex-valued RVQ-VAE audio codec that preserves magnitude-phase coupling across the entire analysis-quantization-synthesis pipeline and removes adversarial discriminators and diffusion post-filters. Without GANs or diffusion, we match or surpass much longer-trained baselines in-domain and reach SOTA out-of-domain performance on phase coherence and waveform fidelity. Compared to standard baselines that train for hundreds of thousands of steps, our model, which reduces the training budget by an order of magnitude, is markedly more compute-efficient while preserving high perceptual quality.

翻译：音频编解码器通过将PCM音频压缩至带宽友好的比特率，为离散音乐生成建模、音乐流媒体和沉浸式媒体提供支持。近期研究倾向于在谱域进行处理；然而，谱图域通常难以处理相位建模，而相位本质上是复数值的。大多数频域神经编解码器要么忽略相位信息，要么将其编码为两个独立的实数值通道，从而限制了空间保真度。这导致需要引入对抗性判别器来补偿音频信号表征能力的不足，但会以收敛速度和训练稳定性为代价。本文提出了一种端到端的复数值RVQ-VAE音频编解码器，它在整个分析-量化-合成流程中保持幅度-相位耦合，并移除了对抗性判别器和扩散后滤波器。在没有GAN或扩散模型的情况下，我们在域内匹配或超越了训练时间长得多的基线模型，并在相位相干性和波形保真度方面达到了域外性能的SOTA水平。与需要训练数十万步的标准基线相比，我们的模型将训练成本降低了一个数量级，在保持高感知质量的同时显著提高了计算效率。