This paper introduces our method for the Emotional Reaction Intensity (ERI) Estimation Challenge, in CVPR 2023: 5th Workshop and Competition on Affective Behavior Analysis in-the-wild (ABAW). Based on the multimodal data provided by the originazers, we extract acoustic and visual features with different pretrained models. The multimodal features are mixed together by Transformer Encoders with cross-modal attention mechnism. In this paper, 1. better features are extracted with the SOTA pretrained models. 2. Compared with the baseline, we improve the Pearson's Correlations Coefficient a lot. 3. We process the data with some special skills to enhance performance ability of our model.
翻译:本文介绍了我们针对CVPR 2023第五届野外情感行为分析研讨会及竞赛(ABAW)中情感反应强度(ERI)估计挑战所提出的方法。基于主办方提供的多模态数据,我们利用不同的预训练模型提取声学特征和视觉特征。通过配备跨模态注意力机制的Transformer编码器对多模态特征进行融合。本文中:(1)采用当前最先进的预训练模型提取更优特征;(2)相较于基线方法,我们显著提升了皮尔逊相关系数;(3)运用特殊数据处理技巧增强模型性能表现。