Music-driven 3D dance generation has become an intensive research topic in recent years with great potential for real-world applications. Most existing methods lack the consideration of genre, which results in genre inconsistency in the generated dance movements. In addition, the correlation between the dance genre and the music has not been investigated. To address these issues, we propose a genre-consistent dance generation framework, GTN-Bailando. First, we propose the Genre Token Network (GTN), which infers the genre from music to enhance the genre consistency of long-term dance generation. Second, to improve the generalization capability of the model, the strategy of pre-training and fine-tuning is adopted.Experimental results on the AIST++ dataset show that the proposed dance generation framework outperforms state-of-the-art methods in terms of motion quality and genre consistency.
翻译:音乐驱动的三维舞蹈生成近年来成为一项重要的研究课题,在实际应用中具有巨大潜力。现有方法大多缺乏对舞蹈流派的考虑,导致生成的舞蹈动作存在流派不一致问题。此外,舞蹈流派与音乐之间的相关性尚未得到充分研究。为解决这些问题,我们提出了一种流派一致的舞蹈生成框架GTN-Bailando。首先,我们设计了流派令牌网络(GTN),该网络从音乐中推断出舞蹈流派,以增强长时舞蹈生成的流派一致性。其次,为提升模型的泛化能力,采用了预训练与微调策略。在AIST++数据集上的实验结果表明,所提出的舞蹈生成框架在动作质量和流派一致性方面均优于现有最先进方法。