Multimodal emotion recognition plays a crucial role in enhancing user experience in human-computer interaction. Over the past few decades, researchers have proposed a series of algorithms and achieved impressive progress. Although each method shows its superior performance, different methods lack a fair comparison due to inconsistencies in feature extractors, evaluation manners, and experimental settings. These inconsistencies severely hinder the development of this field. Therefore, we build MERBench, a unified evaluation benchmark for multimodal emotion recognition. We aim to reveal the contribution of some important techniques employed in previous works, such as feature selection, multimodal fusion, robustness analysis, fine-tuning, pre-training, etc. We hope this benchmark can provide clear and comprehensive guidance for follow-up researchers. Based on the evaluation results of MERBench, we further point out some promising research directions. Additionally, we introduce a new emotion dataset MER2023, focusing on the Chinese language environment. This dataset can serve as a benchmark dataset for research on multi-label learning, noise robustness, and semi-supervised learning. We encourage the follow-up researchers to evaluate their algorithms under the same experimental setup as MERBench for fair comparisons. Our code is available at: https://github.com/zeroQiaoba/MERTools.

翻译：多模态情感识别在人机交互中对于提升用户体验起着关键作用。过去数十年间，研究者提出了一系列算法并取得了显著进展。尽管各方法均展现出优异性能，但由于特征提取器、评估方式及实验设置的不一致性，不同方法之间缺乏公平比较，严重制约了该领域的发展。为此，我们构建了MERBench——一个面向多模态情感识别的统一评估基准。本基准旨在揭示前人工作中关键技术的贡献，如特征选择、多模态融合、鲁棒性分析、微调、预训练等，期望为后续研究者提供清晰全面的指导。基于MERBench评估结果，我们进一步指出了若干具有前景的研究方向。此外，我们引入了面向中文语言环境的新型情感数据集MER2023，可作为多标签学习、噪声鲁棒性及半监督学习研究的基准数据集。我们鼓励后续研究者采用与MERBench相同的实验设置评估其算法，以实现公平比较。代码已开源：https://github.com/zeroQiaoba/MERTools。