Recent increase of remote-work, online meeting and tele-operation task makes people find that gesture for avatars and communication robots is more important than we have thought. It is one of the key factors to achieve smooth and natural communication between humans and AI systems and has been intensively researched. Current gesture generation methods are mostly based on deep neural network using text, audio and other information as the input, however, they generate gestures mainly based on audio, which is called a beat gesture. Although the ratio of the beat gesture is more than 70% of actual human gestures, content based gestures sometimes play an important role to make avatars more realistic and human-like. In this paper, we propose a attention-based contrastive learning for text-to-gesture (ACT2G), where generated gestures represent content of the text by estimating attention weight for each word from the input text. In the method, since text and gesture features calculated by the attention weight are mapped to the same latent space by contrastive learning, once text is given as input, the network outputs a feature vector which can be used to generate gestures related to the content. User study confirmed that the gestures generated by ACT2G were better than existing methods. In addition, it was demonstrated that wide variation of gestures were generated from the same text by changing attention weights by creators.

翻译：近期远程工作、在线会议及远程操作任务的增加，使人们意识到虚拟形象与通信机器人的手势比预想更为重要。手势是实现人机系统间流畅自然通信的关键因素之一，已得到广泛研究。当前手势生成方法大多采用基于文本、音频等信息的深度神经网络，但其主要通过音频生成手势（即节拍手势）。尽管节拍手势在实际人类手势中占比超过70%，但基于内容的手势有时对提升虚拟形象的真实感与类人特性具有重要作用。本文提出了一种基于注意力机制的文本到手势对比学习方法（ACT2G），通过从输入文本中估计每个词的注意力权重，使生成的手势能够表达文本内容。该方法中，通过对比学习将注意力权重计算得到的文本特征与手势特征映射至同一潜在空间，因此当输入文本时，网络能够输出可用于生成与内容相关手势的特征向量。用户研究证实，ACT2G生成的手势优于现有方法。此外，实验表明，创作者可通过调整注意力权重，从同一文本生成多样化的手势。