We address the challenging task of human reaction generation, which aims to generate a corresponding reaction based on an input action. Most of the existing works do not focus on generating and predicting the reaction and cannot generate the motion when only the action is given as input. To address this limitation, we propose a novel interaction Transformer (InterFormer) consisting of a Transformer network with both temporal and spatial attention. Specifically, temporal attention captures the temporal dependencies of the motion of both characters and of their interaction, while spatial attention learns the dependencies between the different body parts of each character and those which are part of the interaction. Moreover, we propose using graphs to increase the performance of spatial attention via an interaction distance module that helps focus on nearby joints from both characters. Extensive experiments on the SBU interaction, K3HI, and DuetDance datasets demonstrate the effectiveness of InterFormer. Our method is general and can be used to generate more complex and long-term interactions. We also provide videos of generated reactions and the code with pre-trained models at https://github.com/CRISTAL-3DSAM/InterFormer

翻译：我们解决了人类反应生成这一具有挑战性的任务，旨在根据输入动作生成相应的反应。现有大多工作未专注于反应的生成与预测，且无法在仅以动作为输入的情况下生成运动。为克服这一局限，我们提出了一种新颖的交互Transformer（InterFormer），它包含一个兼具时间注意力和空间注意力的Transformer网络。具体而言，时间注意力捕捉两个角色运动及其交互的时间依赖关系，而空间注意力则学习每个角色不同身体部位之间以及参与交互部位之间的依赖关系。此外，我们提出利用图结构，通过交互距离模块增强空间注意力性能，该模块有助于聚焦两个角色的邻近关节点。在SBU交互、K3HI和DuetDance数据集上的大量实验证明了InterFormer的有效性。我们的方法具有通用性，可用于生成更复杂、更长期的交互。我们还在https://github.com/CRISTAL-3DSAM/InterFormer提供了生成反应的视频及含预训练模型的代码。