Current methods for creating drum loop audio in digital music production, such as using one-shot samples or resampling, often demand non-trivial efforts of creators. While recent generative models achieve high fidelity and adhere to text, they lack the specific control needed for such a task. Existing symbolic-to-audio research often focuses on single, tonal instruments, leaving the challenge of polyphonic, percussive drum synthesis unaddressed. We address this gap by introducing ``Break-the-Beat!,'' a model capable of rendering a drum MIDI with the timbre of a reference audio. It is built by fine-tuning a pre-trained text-to-audio model with our proposed content encoder and a effective hybrid conditioning mechanism. To enable this, we construct a new dataset of paired target-reference drum audio from existing drum audio datasets. Experiments demonstrate that our model generates high-quality drum audio that follows high-resolution drum MIDI, achieving strong performance across metrics of audio quality, rhythmic alignment, and beat continuity. This offer producers a new, controllable tool for creative production. Demo page: https://ik4sumii.github.io/break-the-beat/

翻译：当前数字音乐制作中创建鼓循环音频的方法（如使用单次采样或重采样）往往需要创作者付出大量精力。尽管近期生成模型已能实现高保真度并遵循文本指令，但此类任务仍缺乏所需的特定控制能力。现有符号到音频的研究通常专注于单一调性乐器，未解决多音色打击乐鼓合成的挑战。我们通过引入"拍打节拍！"模型填补了这一空白，该模型能够根据参考音频的音色渲染鼓MIDI。通过使用我们提出的内容编码器和高效的混合条件机制微调预训练文本到音频模型构建而成。为实现这一目标，我们基于现有鼓音频数据集构建了配对目标-参考鼓音频的新数据集。实验表明，我们的模型能生成遵循高分辨率鼓MIDI的高质量鼓音频，在音频质量、节奏对齐和节拍连续性等指标上均表现优异。这为制作人提供了一种可控的新型创意制作工具。演示页面：https://ik4sumii.github.io/break-the-beat/