Generating the motion of orchestral conductors from a given piece of symphony music is a challenging task since it requires a model to learn semantic music features and capture the underlying distribution of real conducting motion. Prior works have applied Generative Adversarial Networks (GAN) to this task, but the promising diffusion model, which recently showed its advantages in terms of both training stability and output quality, has not been exploited in this context. This paper presents Diffusion-Conductor, a novel DDIM-based approach for music-driven conducting motion generation, which integrates the diffusion model to a two-stage learning framework. We further propose a random masking strategy to improve the feature robustness, and use a pair of geometric loss functions to impose additional regularizations and increase motion diversity. We also design several novel metrics, including Frechet Gesture Distance (FGD) and Beat Consistency Score (BC) for a more comprehensive evaluation of the generated motion. Experimental results demonstrate the advantages of our model.

翻译：从给定的交响乐片段生成管弦乐队指挥的动作是一项具有挑战性的任务，因为它要求模型学习语义音乐特征并捕捉真实指挥动作的潜在分布。先前的研究已采用生成对抗网络（GAN）处理该任务，但近年来在训练稳定性和输出质量方面展现出优势的扩散模型在此背景下尚未得到充分开发。本文提出了Diffusion-Conductor，一种基于DDIM的新型方法，用于音乐驱动的指挥动作生成，该方法将扩散模型集成到两阶段学习框架中。我们进一步提出随机掩码策略以增强特征鲁棒性，并采用一对几何损失函数施加额外正则化、提升动作多样性。同时，我们设计了多项新型评估指标，包括弗雷歇手势距离（FGD）和节拍一致性分数（BC），以实现对生成动作的更全面评估。实验结果表明了本模型的优越性。