This paper describes a system developed for the GENEA (Generation and Evaluation of Non-verbal Behaviour for Embodied Agents) Challenge 2023. Our solution builds on an existing diffusion-based motion synthesis model. We propose a contrastive speech and motion pretraining (CSMP) module, which learns a joint embedding for speech and gesture with the aim to learn a semantic coupling between these modalities. The output of the CSMP module is used as a conditioning signal in the diffusion-based gesture synthesis model in order to achieve semantically-aware co-speech gesture generation. Our entry achieved highest human-likeness and highest speech appropriateness rating among the submitted entries. This indicates that our system is a promising approach to achieve human-like co-speech gestures in agents that carry semantic meaning.

翻译：本文描述了为2023年GENEA（具身智能体非语言行为生成与评估）挑战赛开发的系统。我们的解决方案建立在现有基于扩散的运动合成模型之上。我们提出了对比性语音与运动预训练（CSMP）模块，该模块学习语音与手势的联合嵌入，旨在捕捉这两种模态之间的语义耦合。CSMP模块的输出被用作基于扩散的手势合成模型中的条件信号，以实现语义感知的共语手势生成。在提交的参赛作品中，我们的系统获得了最高的人类相似度和最高的语音适宜性评分。这表明该系统是一种在具身智能体中生成携带语义信息且近似人类动作的共语手势的可行方法。