The ability to automatically generate music that appropriately matches an arbitrary input track is a challenging task. We present a novel controllable system for generating single stems to accompany musical mixes of arbitrary length. At the core of our method are audio autoencoders that efficiently compress audio waveform samples into invertible latent representations, and a conditional latent diffusion model that takes as input the latent encoding of a mix and generates the latent encoding of a corresponding stem. To provide control over the timbre of generated samples, we introduce a technique to ground the latent space to a user-provided reference style during diffusion sampling. For further improving audio quality, we adapt classifier-free guidance to avoid distortions at high guidance strengths when generating an unbounded latent space. We train our model on a dataset of pairs of mixes and matching bass stems. Quantitative experiments demonstrate that, given an input mix, the proposed system can generate basslines with user-specified timbres. Our controllable conditional audio generation framework represents a significant step forward in creating generative AI tools to assist musicians in music production.

翻译：自动生成与任意输入音轨恰当匹配的音乐是一项极具挑战性的任务。我们提出了一种新颖的可控系统，用于生成长度任意的音乐混合音频中的单一声部。本方法的核心是音频自编码器，它能够高效地将音频波形样本压缩为可逆的潜在表示，以及一个条件潜在扩散模型，该模型以混合音频的潜在编码为输入，生成对应声部的潜在编码。为了对生成样本的音色进行控制，我们引入了一种技术，在扩散采样过程中将潜在空间锚定到用户提供的参考风格上。为进一步提升音频质量，我们采用了无分类器引导的改进方法，以避免在生成无界潜在空间时高引导强度导致的失真。我们在由混合音频与其匹配的低音声部所组成的数据集上训练模型。定量实验表明，给定输入混合音频，所提出的系统能生成具有用户指定音色的低音线条。这一可控条件音频生成框架标志着在创建生成式AI工具以辅助音乐制作方面迈出了重要一步。