We present HAFM, a system that generates instrumental music audio to accompany input vocals. Given isolated singing voice, HAFM produces a coherent instrumental accompaniment that can be directly mixed with the input to create complete music. We propose three key innovations over prior work: (1) a dual-rate codec tokenization scheme using HuBERT semantic tokens at 50\,Hz for vocals and EnCodec acoustic tokens at 75\,Hz for instrumentals, enabling time-aligned yet rate-independent modeling; (2) a three-stage hierarchical autoregressive architecture (semantic to coarse acoustic to fine acoustic) with interleaved multi-codebook prediction and classifier-free guidance; and (3) modern Transformer design choices including QK-norm, GEGLU activations, RMSNorm, and T5-style relative position bias for improved training stability and sequence generalization. Experiments on MUSDB18 demonstrate that HAFM achieves a Fréchet Audio Distance (FAD) of 2.08 on isolated vocal inputs, outperforming retrieval baselines and matching prior state-of-the-art systems with fewer parameters. The source code is available at https://github.com/HackerHyper/HAFM.

翻译：我们提出HAFM系统，该系统能生成与输入人声相匹配的乐器音频伴奏。给定分离后的清唱人声，HAFM可生成连贯的乐器伴奏，该伴奏可直接与输入混合以形成完整音乐。相较于现有方法，我们提出三项核心创新：(1) 采用双速率编解码器分词方案，对人声使用50 Hz的HuBERT语义令牌，对乐器声使用75 Hz的EnCodec声学令牌，实现时域对齐且速率无关的建模；(2) 提出三阶段分层自回归架构（语义→粗粒度声学→细粒度声学），结合交错多码本预测与无分类器引导；(3) 采用现代Transformer设计，包括QK归一化、GEGLU激活函数、RMS归一化及T5式相对位置偏置，以提升训练稳定性与序列泛化能力。在MUSDB18数据集上的实验表明，HAFM在清唱人声输入上取得了2.08的弗雷歇音频距离（FAD），在参数更少的情况下优于检索基线方法，并达到与先前最优系统相当的性能。源代码已开源至 https://github.com/HackerHyper/HAFM。