Generating facial reactions in a human-human dyadic interaction is complex and highly dependent on the context since more than one facial reactions can be appropriate for the speaker's behaviour. This has challenged existing machine learning (ML) methods, whose training strategies enforce models to reproduce a specific (not multiple) facial reaction from each input speaker behaviour. This paper proposes the first multiple appropriate facial reaction generation framework that re-formulates the one-to-many mapping facial reaction generation problem as a one-to-one mapping problem. This means that we approach this problem by considering the generation of a distribution of the listener's appropriate facial reactions instead of multiple different appropriate facial reactions, i.e., 'many' appropriate facial reaction labels are summarised as 'one' distribution label during training. Our model consists of a perceptual processor, a cognitive processor, and a motor processor. The motor processor is implemented with a novel Reversible Multi-dimensional Edge Graph Neural Network (REGNN). This allows us to obtain a distribution of appropriate real facial reactions during the training process, enabling the cognitive processor to be trained to predict the appropriate facial reaction distribution. At the inference stage, the REGNN decodes an appropriate facial reaction by using this distribution as input. Experimental results demonstrate that our approach outperforms existing models in generating more appropriate, realistic, and synchronized facial reactions. The improved performance is largely attributed to the proposed appropriate facial reaction distribution learning strategy and the use of a REGNN. The code is available at https://github.com/TongXu-05/REGNN-Multiple-Appropriate-Facial-Reaction-Generation.

翻译：在人类双方面对面交互中生成面部反应是一项复杂且高度依赖于上下文的任务，因为说话者的行为可能对应多种适当的面部反应。这一问题对现有机器学习（ML）方法提出了挑战，因为其训练策略迫使模型从每个输入的说话者行为中重现特定（而非多种）的面部反应。本文首次提出了一种多种适当面部反应生成框架，将一对多映射的面部反应生成问题重新表述为一对一映射问题。这意味着，我们通过考虑生成倾听者适当面部反应的分布（而非多种不同的适当面部反应）来处理该问题，即：在训练过程中，将"多种"适当的面部反应标签归纳为"一种"分布标签。我们的模型由感知处理器、认知处理器和运动处理器组成。运动处理器采用了一种新颖的可逆多维边图神经网络（REGNN）实现。这使得我们能够在训练过程中获得真实适当面部反应的分布，从而训练认知处理器预测适当的反应分布。在推理阶段，REGNN通过将该分布作为输入来解码出适当的单一面部反应。实验结果表明，我们的方法在生成更适当、更真实且更同步的面部反应方面优于现有模型。性能提升主要归功于所提出的适当面部反应分布学习策略以及REGNN的使用。代码开源在https://github.com/TongXu-05/REGNN-Multiple-Appropriate-Facial-Reaction-Generation。