We present ZeroBAS, a neural method to synthesize binaural audio from monaural audio recordings and positional information without training on any binaural data. To our knowledge, this is the first published zero-shot neural approach to mono-to-binaural audio synthesis. Specifically, we show that a parameter-free geometric time warping and amplitude scaling based on source location suffices to get an initial binaural synthesis that can be refined by iteratively applying a pretrained denoising vocoder. Furthermore, we find this leads to generalization across room conditions, which we measure by introducing a new dataset, TUT Mono-to-Binaural, to evaluate state-of-the-art monaural-to-binaural synthesis methods on unseen conditions. Our zero-shot method is perceptually on-par with the performance of supervised methods on the standard mono-to-binaural dataset, and even surpasses them on our out-of-distribution TUT Mono-to-Binaural dataset. Our results highlight the potential of pretrained generative audio models and zero-shot learning to unlock robust binaural audio synthesis.

翻译：我们提出ZeroBAS，一种无需任何双声道数据训练即可从单声道音频录音与位置信息合成双声道音频的神经方法。据我们所知，这是首个公开的零样本神经单声道至双声道音频合成方法。具体而言，我们证明基于声源位置的无参数几何时间扭曲与幅度缩放足以获得初始双声道合成结果，该结果可通过迭代应用预训练的降噪声码器进行优化。此外，我们发现该方法能够实现跨房间条件的泛化能力，为此我们引入新数据集TUT Mono-to-Binaural进行评估，在未见条件下测试最先进的单声道至双声道合成方法。我们的零样本方法在标准单声道至双声道数据集上的感知性能与监督学习方法相当，甚至在我们分布外TUT Mono-to-Binaural数据集上超越后者。研究结果凸显了预训练生成音频模型与零样本学习在实现鲁棒双声道音频合成方面的潜力。