The study of music-generated dance is a novel and challenging Image generation task. It aims to input a piece of music and seed motions, then generate natural dance movements for the subsequent music. Transformer-based methods face challenges in time series prediction tasks related to human movements and music due to their struggle in capturing the nonlinear relationship and temporal aspects. This can lead to issues like joint deformation, role deviation, floating, and inconsistencies in dance movements generated in response to the music. In this paper, we propose a Quaternion-Enhanced Attention Network (QEAN) for visual dance synthesis from a quaternion perspective, which consists of a Spin Position Embedding (SPE) module and a Quaternion Rotary Attention (QRA) module. First, SPE embeds position information into self-attention in a rotational manner, leading to better learning of features of movement sequences and audio sequences, and improved understanding of the connection between music and dance. Second, QRA represents and fuses 3D motion features and audio features in the form of a series of quaternions, enabling the model to better learn the temporal coordination of music and dance under the complex temporal cycle conditions of dance generation. Finally, we conducted experiments on the dataset AIST++, and the results show that our approach achieves better and more robust performance in generating accurate, high-quality dance movements. Our source code and dataset can be available from https://github.com/MarasyZZ/QEAN and https://google.github.io/aistplusplus_dataset respectively.

翻译：音乐驱动的舞蹈生成是一项新颖且具有挑战性的图像生成任务，旨在输入一段音乐及初始动作，为后续音乐生成自然的舞蹈动作。基于Transformer的方法在捕捉人体运动与音乐相关的非线性关系及时间维度方面存在困难，导致在音乐响应中生成的舞蹈动作出现关节变形、角色偏移、漂浮及动作不连贯等问题。本文从四元数视角提出一种用于视觉舞蹈合成的四元数增强注意力网络（QEAN），该网络包含自旋位置嵌入（SPE）模块和四元数旋转注意力（QRA）模块。首先，SPE以旋转方式将位置信息嵌入自注意力机制，更好地学习运动序列和音频序列的特征，增强对音乐与舞蹈之间联系的理解。其次，QRA以四元数序列形式表示并融合三维运动特征与音频特征，使模型在舞蹈生成的复杂时间循环条件下更好地学习音乐与舞蹈的时间协调性。最后，我们在AIST++数据集上进行了实验，结果表明，我们的方法在生成精确、高质量的舞蹈动作方面取得了更优且更鲁棒的性能。我们的源代码和数据集可分别从 https://github.com/MarasyZZ/QEAN 和 https://google.github.io/aistplusplus_dataset 获取。