In the field of affective computing, researchers in the community have promoted the performance of models and algorithms by using the complementarity of multimodal information. However, the emergence of more and more modal information makes the development of datasets unable to keep up with the progress of existing modal sensing equipment. Collecting and studying multimodal data is a complex and significant work. In order to supplement the challenge of partial missing of community data. We collected and labeled a new homogeneous multimodal gesture emotion recognition dataset based on the analysis of the existing data sets. This data set complements the defects of homogeneous multimodal data and provides a new research direction for emotion recognition. Moreover, we propose a pseudo dual-flow network based on this dataset, and verify the application potential of this dataset in the affective computing community. The experimental results demonstrate that it is feasible to use the traditional visual information and spiking visual information based on homogeneous multimodal data for visual emotion recognition.The dataset is available at \url{https://github.com/201528014227051/SGED}
翻译:在情感计算领域,研究人员利用多模态信息的互补性提升了模型与算法的性能。然而,随着越来越多模态信息的涌现,数据集的发展已无法跟上现有模态传感设备的进步。收集和研究多模态数据是一项复杂而重要的工作。为补充社区数据部分缺失的挑战,我们在分析现有数据集的基础上,收集并标注了一个新的同质多模态手势情感识别数据集。该数据集弥补了同质多模态数据存在的缺陷,并为情感识别提供了新的研究方向。此外,我们基于该数据集提出了一个伪双流网络,并验证了该数据集在情感计算社区的应用潜力。实验结果表明,基于同质多模态数据,利用传统视觉信息与脉冲视觉信息进行视觉情感识别是可行的。该数据集可在 \url{https://github.com/201528014227051/SGED} 获取。