Generating the motion of orchestral conductors from a given piece of symphony music is a challenging task since it requires a model to learn semantic music features and capture the underlying distribution of real conducting motion. Prior works have applied Generative Adversarial Networks (GAN) to this task, but the promising diffusion model, which recently showed its advantages in terms of both training stability and output quality, has not been exploited in this context. This paper presents Diffusion-Conductor, a novel DDIM-based approach for music-driven conducting motion generation, which integrates the diffusion model to a two-stage learning framework. We further propose a random masking strategy to improve the feature robustness, and use a pair of geometric loss functions to impose additional regularizations and increase motion diversity. We also design several novel metrics, including Frechet Gesture Distance (FGD) and Beat Consistency Score (BC) for a more comprehensive evaluation of the generated motion. Experimental results demonstrate the advantages of our model.

翻译：从一段给定的交响乐乐曲中生成管弦乐队指挥的动作是一项具有挑战性的任务，因为这要求模型学习语义化的音乐特征，并捕捉真实指挥动作的潜在分布。先前的工作已将生成对抗网络（GAN）应用于该任务，但近期在训练稳定性和输出质量方面展现出优势的扩散模型，尚未在此领域得到应用。本文提出Diffusion-Conductor，一种基于DDIM的新型音乐驱动指挥动作生成方法，该方法将扩散模型集成到一个两阶段学习框架中。我们进一步提出随机掩蔽策略以增强特征鲁棒性，并利用一对几何损失函数施加额外的正则化，提高动作多样性。我们还设计了若干新型评估指标，包括弗雷歇手势距离（FGD）和节拍一致性得分（BC），以更全面地评估生成的动作。实验结果表明了我们模型的优越性。