Stance detection automatically detects the stance in a text towards a target, vital for content analysis in web and social media research. Despite their promising capabilities, LLMs encounter challenges when directly applied to stance detection. First, stance detection demands multi-aspect knowledge, from deciphering event-related terminologies to understanding the expression styles in social media platforms. Second, stance detection requires advanced reasoning to infer authors' implicit viewpoints, as stance are often subtly embedded rather than overtly stated in the text. To address these challenges, we design a three-stage framework COLA (short for Collaborative rOle-infused LLM-based Agents) in which LLMs are designated distinct roles, creating a collaborative system where each role contributes uniquely. Initially, in the multidimensional text analysis stage, we configure the LLMs to act as a linguistic expert, a domain specialist, and a social media veteran to get a multifaceted analysis of texts, thus overcoming the first challenge. Next, in the reasoning-enhanced debating stage, for each potential stance, we designate a specific LLM-based agent to advocate for it, guiding the LLM to detect logical connections between text features and stance, tackling the second challenge. Finally, in the stance conclusion stage, a final decision maker agent consolidates prior insights to determine the stance. Our approach avoids extra annotated data and model training and is highly usable. We achieve state-of-the-art performance across multiple datasets. Ablation studies validate the effectiveness of each design role in handling stance detection. Further experiments have demonstrated the explainability and the versatility of our approach. Our approach excels in usability, accuracy, effectiveness, explainability and versatility, highlighting its value.
翻译:立场检测自动检测文本中针对某一目标的立场,对于网络与社交媒体研究中的内容分析至关重要。尽管大语言模型(LLM)展现出强大能力,但在直接应用于立场检测时仍面临挑战。首先,立场检测需要多方面的知识——从解读事件相关术语到理解社交媒体平台的表达风格。其次,立场检测要求具备高级推理能力以推断作者的隐含观点,因为立场往往以微妙方式嵌入文本,而非直接陈述。为应对这些挑战,我们设计了一个三阶段框架COLA(Collaborative rOle-infused LLM-based Agents的缩写),其中LLM被赋予不同角色,构建协作系统,每个角色贡献独特功能。首先,在多维文本分析阶段,我们配置LLM分别扮演语言专家、领域专家和社交媒体资深用户,对文本进行多角度分析,从而克服第一项挑战。其次,在推理增强辩论阶段,针对每种潜在立场,我们指定特定的基于LLM的智能体为之辩护,引导LLM发现文本特征与立场之间的逻辑关联,解决第二项挑战。最后,在立场结论阶段,由最终决策者智能体整合先前见解以确定立场。我们的方法无需额外标注数据和模型训练,且具有高度可用性。我们在多个数据集上实现了最优性能。消融研究验证了各设计角色在处理立场检测中的有效性。进一步实验证明了我们方法的可解释性与通用性。该方法在可用性、准确性、有效性、可解释性及通用性方面表现卓越,彰显其价值。