Over the past few decades, multimodal emotion recognition has made remarkable progress with the development of deep learning. However, existing technologies are difficult to meet the demand for practical applications. To improve the robustness, we launch a Multimodal Emotion Recognition Challenge (MER 2023) to motivate global researchers to build innovative technologies that can further accelerate and foster research. For this year's challenge, we present three distinct sub-challenges: (1) MER-MULTI, in which participants recognize both discrete and dimensional emotions; (2) MER-NOISE, in which noise is added to test videos for modality robustness evaluation; (3) MER-SEMI, which provides large amounts of unlabeled samples for semi-supervised learning. In this paper, we test a variety of multimodal features and provide a competitive baseline for each sub-challenge. Our system achieves 77.57% on the F1 score and 0.82 on the mean squared error (MSE) for MER-MULTI, 69.82% on the F1 score and 1.12 on MSE for MER-NOISE, and 86.75% on the F1 score for MER-SEMI, respectively. Baseline code is available at https://github.com/zeroQiaoba/MER2023-Baseline.
翻译:在过去的几十年中,随着深度学习的发展,多模态情感识别取得了显著进展。然而,现有技术难以满足实际应用需求。为提升鲁棒性,我们发起了2023年多模态情感识别挑战赛(MER 2023),旨在激励全球研究人员开发创新技术,从而进一步加速和推动相关研究。针对本年度挑战,我们设立了三个独特的子任务:(1)MER-MULTI,要求参与者同时识别离散情感和维度情感;(2)MER-NOISE,在测试视频中添加噪声以评估模态鲁棒性;(3)MER-SEMI,提供大量无标签样本用于半监督学习。本文测试了多种多模态特征,并为每个子任务提供了具有竞争力的基线系统。我们的系统在MER-MULTI上实现了77.57%的F1分数和0.82的均方误差(MSE),在MER-NOISE上实现了69.82%的F1分数和1.12的MSE,在MER-SEMI上实现了86.75%的F1分数。基线代码可从https://github.com/zeroQiaoba/MER2023-Baseline获取。