Singing Accompaniment Generation (SAG), which generates instrumental music to accompany input vocals, is crucial to developing human-AI symbiotic art creation systems. The state-of-the-art method, SingSong, utilizes a multi-stage autoregressive (AR) model for SAG, however, this method is extremely slow as it generates semantic and acoustic tokens recursively, and this makes it impossible for real-time applications. In this paper, we aim to develop a Fast SAG method that can create high-quality and coherent accompaniments. A non-AR diffusion-based framework is developed, which by carefully designing the conditions inferred from the vocal signals, generates the Mel spectrogram of the target accompaniment directly. With diffusion and Mel spectrogram modeling, the proposed method significantly simplifies the AR token-based SingSong framework, and largely accelerates the generation. We also design semantic projection, prior projection blocks as well as a set of loss functions, to ensure the generated accompaniment has semantic and rhythm coherence with the vocal signal. By intensive experimental studies, we demonstrate that the proposed method can generate better samples than SingSong, and accelerate the generation by at least 30 times. Audio samples and code are available at https://fastsag.github.io/.

翻译：歌唱伴奏生成（SAG）通过生成器乐来匹配输入的人声，对于发展人机共生艺术创作系统至关重要。当前最先进的方法SingSong采用多阶段自回归（AR）模型进行SAG，但该方法因递归生成语义和声学标记而极其缓慢，无法支持实时应用。本文旨在开发一种快速SAG方法，以生成高质量且连贯的伴奏。我们提出了一种基于非自回归扩散的框架，通过精心设计从人声信号中推断的条件，直接生成目标伴奏的梅尔频谱图。借助扩散模型与梅尔频谱图建模，所提方法显著简化了基于AR标记的SingSong框架，并大幅提升了生成速度。我们还设计了语义投影块、先验投影块以及一组损失函数，以确保生成的伴奏与人声信号在语义和节奏上保持连贯。通过大量实验研究表明，所提方法能生成优于SingSong的样本，并将生成速度提升至少30倍。音频样本和代码已开源至https://fastsag.github.io/。