The art of communication beyond speech there are gestures. The automatic co-speech gesture generation draws much attention in computer animation. It is a challenging task due to the diversity of gestures and the difficulty of matching the rhythm and semantics of the gesture to the corresponding speech. To address these problems, we present DiffuseStyleGesture, a diffusion model based speech-driven gesture generation approach. It generates high-quality, speech-matched, stylized, and diverse co-speech gestures based on given speeches of arbitrary length. Specifically, we introduce cross-local attention and self-attention to the gesture diffusion pipeline to generate better speech matched and realistic gestures. We then train our model with classifier-free guidance to control the gesture style by interpolation or extrapolation. Additionally, we improve the diversity of generated gestures with different initial gestures and noise. Extensive experiments show that our method outperforms recent approaches on speech-driven gesture generation. Our code, pre-trained models, and demos are available at https://github.com/YoungSeng/DiffuseStyleGesture.

翻译：超越言语的沟通艺术——手势。计算机动画领域中，自动共语手势生成备受关注。由于手势的多样性，以及手势节奏与语义难以与对应的语音相匹配，该任务极具挑战性。为解决这些问题，我们提出了DiffuseStyleGesture——一种基于扩散模型的语音驱动手势生成方法。它能根据任意长度的给定语音，生成高质量、与语音匹配、有风格且多样化的共语手势。具体而言，我们在手势扩散流程中引入了交叉局部注意力和自注意力，以生成与语音匹配更佳、更逼真的手势。随后，我们通过无分类器指导训练模型，利用插值或外推方式控制手势风格。此外，我们通过不同初始手势和噪声提升了生成手势的多样性。大量实验表明，我们的方法在语音驱动手势生成方面优于近期其他方法。我们的代码、预训练模型及演示可在https://github.com/YoungSeng/DiffuseStyleGesture获取。