Animating virtual avatars to make co-speech gestures facilitates various applications in human-machine interaction. The existing methods mainly rely on generative adversarial networks (GANs), which typically suffer from notorious mode collapse and unstable training, thus making it difficult to learn accurate audio-gesture joint distributions. In this work, we propose a novel diffusion-based framework, named Diffusion Co-Speech Gesture (DiffGesture), to effectively capture the cross-modal audio-to-gesture associations and preserve temporal coherence for high-fidelity audio-driven co-speech gesture generation. Specifically, we first establish the diffusion-conditional generation process on clips of skeleton sequences and audio to enable the whole framework. Then, a novel Diffusion Audio-Gesture Transformer is devised to better attend to the information from multiple modalities and model the long-term temporal dependency. Moreover, to eliminate temporal inconsistency, we propose an effective Diffusion Gesture Stabilizer with an annealed noise sampling strategy. Benefiting from the architectural advantages of diffusion models, we further incorporate implicit classifier-free guidance to trade off between diversity and gesture quality. Extensive experiments demonstrate that DiffGesture achieves state-of-theart performance, which renders coherent gestures with better mode coverage and stronger audio correlations. Code is available at https://github.com/Advocate99/DiffGesture.

翻译：为虚拟化身制作共语手势动画有助于人机交互中的多种应用。现有方法主要依赖生成对抗网络（GANs），但这类方法通常面临众所周知的模式坍塌和训练不稳定问题，难以学习准确的音频-手势联合分布。在本文中，我们提出了一种名为扩散共语手势（DiffGesture）的新型扩散框架，旨在有效捕捉跨模态的音频到手势关联并保持时间连贯性，以实现高保真度的音频驱动共语手势生成。具体而言，我们首先在骨架序列和音频片段上建立扩散条件生成过程，以支撑整个框架。接着设计了创新的扩散音频-手势变换器，更好地关注来自多模态的信息并建模长期时间依赖性。此外，为消除时间不一致性，我们提出一种带有退火噪声采样策略的有效扩散手势稳定器。得益于扩散模型的结构优势，我们进一步整合隐式无分类器引导，以在多样性与手势质量之间取得平衡。大量实验表明，DiffGesture达到了最先进的性能，能够生成具有更好模式覆盖和更强音频关联性的连贯手势。代码已开源在 https://github.com/Advocate99/DiffGesture。