Dance and music are closely related forms of expression, with mutual retrieval between dance videos and music being a fundamental task in various fields like education, art, and sports. However, existing methods often suffer from unnatural generation effects or fail to fully explore the correlation between music and dance. To overcome these challenges, we propose BeatDance, a novel beat-based model-agnostic contrastive learning framework. BeatDance incorporates a Beat-Aware Music-Dance InfoExtractor, a Trans-Temporal Beat Blender, and a Beat-Enhanced Hubness Reducer to improve dance-music retrieval performance by utilizing the alignment between music beats and dance movements. We also introduce the Music-Dance (MD) dataset, a large-scale collection of over 10,000 music-dance video pairs for training and testing. Experimental results on the MD dataset demonstrate the superiority of our method over existing baselines, achieving state-of-the-art performance. The code and dataset will be made public available upon acceptance.
翻译:舞蹈与音乐是密切相关的表达形式,两者之间的相互检索是教育、艺术、体育等多个领域的基础任务。然而,现有方法常存在生成效果不自然的问题,或未能充分挖掘音乐与舞蹈之间的关联。为克服这些挑战,我们提出BeatDance——一种新颖的基于节拍的模型无关对比学习框架。该框架包含节拍感知音乐-舞蹈信息提取器、跨时间节拍混合器以及节拍增强集散度缩减器,通过利用音乐节拍与舞蹈动作的对齐关系提升舞蹈-音乐检索性能。我们还构建了音乐-舞蹈数据集MD,包含超10,000对音乐-舞蹈视频的大规模训练测试集合。在MD数据集上的实验结果表明,本方法优于现有基线方法,实现了最先进的性能。相关代码与数据集将在论文录用后公开。