Advances in generative models and sequence learning have greatly promoted research in dance motion generation, yet current methods still suffer from coarse semantic control and poor coherence in long sequences. In this work, we present Listen to Rhythm, Choose Movements (LRCM), a multimodal-guided diffusion framework supporting both diverse input modalities and autoregressive dance motion generation. We explore a feature decoupling paradigm for dance datasets and generalize it to the Motorica Dance dataset, separating motion capture data, audio rhythm, and professionally annotated global and local text descriptions. Our diffusion architecture integrates an audio-latent Conformer and a text-latent Cross-Conformer, and incorporates a Motion Temporal Mamba Module (MTMM) to enable smooth, long-duration autoregressive synthesis. Experimental results indicate that LRCM delivers strong performance in both functional capability and quantitative metrics, demonstrating notable potential in multimodal input scenarios and extended sequence generation. The project page is available at https://oranduanstudy.github.io/LRCM/.
翻译:生成模型与序列学习的进步极大地推动了舞蹈动作生成的研究,然而当前方法仍存在语义控制粗粒度、长序列连贯性差等问题。本文提出"听韵律,择舞步"(Listen to Rhythm, Choose Movements, LRCM)——一个支持多样化输入模态与自回归舞蹈动作生成的多模态引导扩散框架。我们探索了舞蹈数据集的特征解耦范式,并将其泛化至Motorica舞蹈数据集,实现了动作捕捉数据、音频节奏以及专业标注的全局与局部文本描述的解耦。我们的扩散架构集成了音频潜变量Conformer与文本潜变量Cross-Conformer,并引入运动时间流形模块(Motion Temporal Mamba Module, MTMM),以实现平滑的长时自回归合成。实验结果表明,LRCM在功能能力与量化指标上均展现出强劲性能,在多模态输入场景与长序列生成方面彰显出显著潜力。项目页面请见https://oranduanstudy.github.io/LRCM/。