We present Subtractive Training, a simple and novel method for synthesizing individual musical instrument stems given other instruments as context. This method pairs a dataset of complete music mixes with 1) a variant of the dataset lacking a specific stem, and 2) LLM-generated instructions describing how the missing stem should be reintroduced. We then fine-tune a pretrained text-to-audio diffusion model to generate the missing instrument stem, guided by both the existing stems and the text instruction. Our results demonstrate Subtractive Training's efficacy in creating authentic drum stems that seamlessly blend with the existing tracks. We also show that we can use the text instruction to control the generation of the inserted stem in terms of rhythm, dynamics, and genre, allowing us to modify the style of a single instrument in a full song while keeping the remaining instruments the same. Lastly, we extend this technique to MIDI formats, successfully generating compatible bass, drum, and guitar parts for incomplete arrangements.

翻译：本文提出了一种简单新颖的减损训练方法，用于在给定其他乐器作为上下文的情况下合成独立的乐器音轨。该方法将完整音乐混音数据集与以下两种数据配对：1) 缺失特定音轨的数据集变体，以及2) 由大语言模型生成的描述如何重新引入缺失音轨的指令文本。随后，我们通过微调预训练的文本到音频扩散模型，使其在现有音轨和文本指令的共同引导下生成缺失的乐器音轨。实验结果表明，减损训练能够有效创建与现有音轨无缝融合的真实鼓组音轨。我们还证明，可通过文本指令从节奏、动态和流派维度控制生成音轨的特性，从而实现在保持其他乐器不变的前提下修改完整歌曲中单一乐器的演奏风格。最后，我们将该技术扩展至MIDI格式，成功为不完整的编曲生成了兼容的贝斯、鼓组和吉他声部。