Audio-driven co-speech human gesture generation has made remarkable advancements recently. However, most previous works only focus on single person audio-driven gesture generation. We aim at solving the problem of conversational co-speech gesture generation that considers multiple participants in a conversation, which is a novel and challenging task due to the difficulty of simultaneously incorporating semantic information and other relevant features from both the primary speaker and the interlocutor. To this end, we propose CoDiffuseGesture, a diffusion model-based approach for speech-driven interaction gesture generation via modeling bilateral conversational intention, emotion, and semantic context. Our method synthesizes appropriate interactive, speech-matched, high-quality gestures for conversational motions through the intention perception module and emotion reasoning module at the sentence level by a pretrained language model. Experimental results demonstrate the promising performance of the proposed method.

翻译：近年来，基于音频驱动的共语人体手势生成取得了显著进展。然而，现有研究大多仅聚焦于单人音频驱动的手势生成。本文旨在解决考虑对话中多方参与者的对话式共语手势生成问题，这是一项新颖且具有挑战性的任务，原因在于需要同时融合主要说话者和对话者的语义信息及相关特征。为此，我们提出CoDiffuseGesture——一种基于扩散模型的方法，通过建模双边对话意图、情感和语义上下文实现语音驱动的交互式手势生成。该方法通过预训练语言模型在句子层面设置意图感知模块和情感推理模块，合成与对话匹配且质量上乘的交互式手势。实验结果表明，该方法具有优越的性能。