Stance detection, a key task in natural language processing, determines an author's viewpoint based on textual analysis. This study evaluates the evolution of stance detection methods, transitioning from early machine learning approaches to the groundbreaking BERT model, and eventually to modern Large Language Models (LLMs) such as ChatGPT, LLaMa-2, and Mistral-7B. While ChatGPT's closed-source nature and associated costs present challenges, the open-source models like LLaMa-2 and Mistral-7B offers an encouraging alternative. Initially, our research focused on fine-tuning ChatGPT, LLaMa-2, and Mistral-7B using several publicly available datasets. Subsequently, to provide a comprehensive comparison, we assess the performance of these models in zero-shot and few-shot learning scenarios. The results underscore the exceptional ability of LLMs in accurately detecting stance, with all tested models surpassing existing benchmarks. Notably, LLaMa-2 and Mistral-7B demonstrate remarkable efficiency and potential for stance detection, despite their smaller sizes compared to ChatGPT. This study emphasizes the potential of LLMs in stance detection and calls for more extensive research in this field.
翻译:立场检测作为自然语言处理中的关键任务,旨在通过文本分析确定作者的立场观点。本研究评估了立场检测方法的演进历程,涵盖从早期机器学习方法到突破性BERT模型,最终延伸至ChatGPT、LLaMa-2和Mistral-7B等现代大语言模型(LLMs)的技术路径。尽管ChatGPT的闭源特性与使用成本带来挑战,但LLaMa-2和Mistral-7B等开源模型提供了富有前景的替代方案。初始阶段,我们利用多个公开数据集对ChatGPT、LLaMa-2和Mistral-7B进行微调。随后,为进行全面比较,我们评估这些模型在零样本和少样本学习场景中的性能表现。结果表明,所有测试模型均超越现有基准,充分彰显大语言模型在准确检测立场方面的卓越能力。值得注意的是,相较于ChatGPT,规模更小的LLaMa-2和Mistral-7B在立场检测中展现出显著效能与发展潜力。本研究强调了大语言模型在立场检测中的应用价值,并呼吁在该领域开展更深入的研究。