Emotional Reaction Intensity(ERI) estimation is an important task in multimodal scenarios, and has fundamental applications in medicine, safe driving and other fields. In this paper, we propose a solution to the ERI challenge of the fifth Affective Behavior Analysis in-the-wild(ABAW), a dual-branch based multi-output regression model. The spatial attention is used to better extract visual features, and the Mel-Frequency Cepstral Coefficients technology extracts acoustic features, and a method named modality dropout is added to fusion multimodal features. Our method achieves excellent results on the official validation set.
翻译:情感反应强度(Emotional Reaction Intensity,ERI)估计是多模态场景中的一项重要任务,在医学、安全驾驶等领域具有基础性应用。本文针对第五届野外情感行为分析(ABAW)竞赛中的ERI挑战,提出了一种基于双分支的多输出回归模型解决方案。该模型利用空间注意力机制更好地提取视觉特征,采用梅尔频率倒谱系数技术提取声学特征,并引入一种称为模态丢弃(modality dropout)的方法来融合多模态特征。我们的方法在官方验证集上取得了优异的结果。