Recent work has made a preliminary attempt to use large language models (LLMs) to solve the stance detection task, showing promising results. However, considering that stance detection usually requires detailed background knowledge, the vanilla reasoning method may neglect the domain knowledge to make a professional and accurate analysis. Thus, there is still room for improvement of LLMs reasoning, especially in leveraging the generation capability of LLMs to simulate specific experts (i.e., multi-agents) to detect the stance. In this paper, different from existing multi-agent works that require detailed descriptions and use fixed experts, we propose a Dynamic Experienced Expert Modeling (DEEM) method which can leverage the generated experienced experts and let LLMs reason in a semi-parametric way, making the experts more generalizable and reliable. Experimental results demonstrate that DEEM consistently achieves the best results on three standard benchmarks, outperforms methods with self-consistency reasoning, and reduces the bias of LLMs.
翻译:近期研究已初步尝试利用大型语言模型解决立场检测任务,展现出可观前景。然而,鉴于立场检测通常需要详尽的背景知识,传统的推理方法可能忽视领域知识,难以做出专业准确的分析。因此,大型语言模型的推理能力仍有提升空间,特别是在利用其生成能力模拟特定专家(即多智能体)进行立场检测方面。本文不同于现有需要详细描述且使用固定专家的多智能体方法,提出了一种动态经验专家建模方法,该方法能够利用生成的经验专家,并以半参数化方式引导大型语言模型进行推理,使专家更具泛化性和可靠性。实验结果表明,DEEM在三个标准基准测试中持续取得最佳结果,优于采用自洽推理的方法,并有效减少了大型语言模型的偏差。