Emotion Recognition in Conversation (ERC) plays a crucial role in enabling dialogue systems to effectively respond to user requests. The emotions in a conversation can be identified by the representations from various modalities, such as audio, visual, and text. However, due to the weak contribution of non-verbal modalities to recognize emotions, multimodal ERC has always been considered a challenging task. In this paper, we propose Teacher-leading Multimodal fusion network for ERC (TelME). TelME incorporates cross-modal knowledge distillation to transfer information from a language model acting as the teacher to the non-verbal students, thereby optimizing the efficacy of the weak modalities. We then combine multimodal features using a shifting fusion approach in which student networks support the teacher. TelME achieves state-of-the-art performance in MELD, a multi-speaker conversation dataset for ERC. Finally, we demonstrate the effectiveness of our components through additional experiments.

翻译：对话情绪识别（ERC）在使对话系统有效响应用户请求中发挥着关键作用。对话中的情绪可通过音频、视觉和文本等多种模态的表征来识别。然而，由于非语言模态对情绪识别的贡献较弱，多模态ERC一直被认为是一项具有挑战性的任务。本文提出了一种用于ERC的教师主导多模态融合网络（TelME）。TelME引入跨模态知识蒸馏，将作为教师的语言模型中的信息传递给非语言学生模型，从而优化弱模态的有效性。随后，我们采用一种学生网络支持教师的移位融合方法来整合多模态特征。TelME在用于ERC的多说话人对话数据集MELD上达到了最先进的性能。最后，通过额外实验验证了各组件的有效性。