Animating virtual avatars to make co-speech gestures facilitates various applications in human-machine interaction. The existing methods mainly rely on generative adversarial networks (GANs), which typically suffer from notorious mode collapse and unstable training, thus making it difficult to learn accurate audio-gesture joint distributions. In this work, we propose a novel diffusion-based framework, named Diffusion Co-Speech Gesture (DiffGesture), to effectively capture the cross-modal audio-to-gesture associations and preserve temporal coherence for high-fidelity audio-driven co-speech gesture generation. Specifically, we first establish the diffusion-conditional generation process on clips of skeleton sequences and audio to enable the whole framework. Then, a novel Diffusion Audio-Gesture Transformer is devised to better attend to the information from multiple modalities and model the long-term temporal dependency. Moreover, to eliminate temporal inconsistency, we propose an effective Diffusion Gesture Stabilizer with an annealed noise sampling strategy. Benefiting from the architectural advantages of diffusion models, we further incorporate implicit classifier-free guidance to trade off between diversity and gesture quality. Extensive experiments demonstrate that DiffGesture achieves state-of-theart performance, which renders coherent gestures with better mode coverage and stronger audio correlations. Code is available at https://github.com/Advocate99/DiffGesture.

翻译：为虚拟化身赋予共语手势动画能力，可促进人机交互领域的多元应用。现有方法主要依赖生成对抗网络（GANs），但此类方法常受困于模式坍塌与训练不稳定问题，难以精确学习音频-手势联合分布。本文提出一种基于扩散模型的新型框架——扩散共语手势模型（DiffGesture），通过有效捕捉跨模态音频-手势关联并保持时序连贯性，实现高保真音频驱动共语手势生成。具体而言，我们首先在骨架序列片段与音频上建立扩散条件生成过程，构建整体框架；其次设计新型扩散音频-手势变换器，以增强多模态信息关注能力并建模长程时序依赖；此外，为消除时序不一致性，提出带有退火噪声采样策略的扩散手势稳定器。得益于扩散模型架构优势，我们进一步引入无分类器隐式引导机制，在多样性与手势质量间取得平衡。大量实验表明，DiffGesture达到最先进性能，生成的连贯手势具有更优模式覆盖与更强的音频相关性。代码开源地址：https://github.com/Advocate99/DiffGesture