Large language models (LLMs) and their variants have shown extraordinary efficacy across numerous downstream natural language processing (NLP) tasks, which has presented a new vision for the development of NLP. Despite their remarkable performance in natural language generating (NLG), LLMs lack a distinct focus on the emotion understanding domain. As a result, using LLMs for emotion recognition may lead to suboptimal and inadequate precision. Another limitation of LLMs is that they are typical trained without leveraging multi-modal information. To overcome these limitations, we propose DialogueLLM, a context and emotion knowledge tuned LLM that is obtained by fine-tuning LLaMA models with 13,638 multi-modal (i.e., texts and videos) emotional dialogues. The visual information is considered as the supplementary knowledge to construct high-quality instructions. We offer a comprehensive evaluation of our proposed model on three benchmarking emotion recognition in conversations (ERC) datasets and compare the results against the SOTA baselines and other SOTA LLMs. Additionally, DialogueLLM-7B can be easily trained using LoRA on a 40GB A100 GPU in 5 hours, facilitating reproducibility for other researchers.

翻译：大语言模型及其变体已在众多自然语言处理下游任务中展现出卓越效能，为自然语言处理的发展提供了新视野。尽管大语言模型在自然语言生成方面表现优异，但其在情感理解领域的针对性关注仍显不足。因此，直接运用大语言模型进行情感识别可能导致结果次优且精度不足。另一局限性在于，大语言模型通常缺乏多模态信息的训练支撑。为突破这些瓶颈，我们提出DialogueLLM——一种经过上下文与情感知识调优的大语言模型，该模型通过使用13,638组多模态（即文本与视频）情感对话对LLaMA模型进行微调而得。视觉信息被视作构建高质量指令的补充知识。我们在三个基准对话情感识别数据集上对所提模型进行了全面评估，并将其结果与当前最优基线及其他先进大语言模型进行对比。此外，DialogueLLM-7B模型仅需在40GB显存的A100 GPU上通过LoRA方法训练5小时即可完成，便于其他研究者复现。