Recent work has made a preliminary attempt to use large language models (LLMs) to solve the stance detection task, showing promising results. However, considering that stance detection usually requires detailed background knowledge, the vanilla reasoning method may neglect the domain knowledge to make a professional and accurate analysis. Thus, there is still room for improvement of LLMs reasoning, especially in leveraging the generation capability of LLMs to simulate specific experts (i.e., multi-agents) to detect the stance. In this paper, different from existing multi-agent works that require detailed descriptions and use fixed experts, we propose a Dynamic Experienced Expert Modeling (DEEM) method which can leverage the generated experienced experts and let LLMs reason in a semi-parametric way, making the experts more generalizable and reliable. Experimental results demonstrate that DEEM consistently achieves the best results on three standard benchmarks, outperforms methods with self-consistency reasoning, and reduces the bias of LLMs.
翻译:近期研究已初步尝试利用大型语言模型解决立场检测任务,并展现出令人瞩目的效果。然而,鉴于立场检测通常需要详尽的背景知识,常规推理方法可能因忽略领域知识而难以进行专业精准的分析。因此,大型语言模型的推理能力仍有提升空间——尤其是通过发挥其生成能力来模拟特定专家(即多智能体)以检测立场。不同于现有需要详细描述且使用固定专家的多智能体方法,本文提出一种动态经验专家建模方法(DEEM),该方法能够利用生成的经验专家,并以半参数化方式驱动大语言模型进行推理,从而使专家模型更具泛化性与可靠性。实验结果表明,DEEM在三个标准基准测试中均取得最佳效果,性能优于采用自一致性推理的方法,并有效降低了语言模型的固有偏差。