Stance detection refers to the task of extracting the standpoint (Favor, Against or Neither) towards a target in given texts. Such research gains increasing attention with the proliferation of social media contents. The conventional framework of handling stance detection is converting it into text classification tasks. Deep learning models have already replaced rule-based models and traditional machine learning models in solving such problems. Current deep neural networks are facing two main challenges which are insufficient labeled data and information in social media posts and the unexplainable nature of deep learning models. A new pre-trained language model chatGPT was launched on Nov 30, 2022. For the stance detection tasks, our experiments show that ChatGPT can achieve SOTA or similar performance for commonly used datasets including SemEval-2016 and P-Stance. At the same time, ChatGPT can provide explanation for its own prediction, which is beyond the capability of any existing model. The explanations for the cases it cannot provide classification results are especially useful. ChatGPT has the potential to be the best AI model for stance detection tasks in NLP, or at least change the research paradigm of this field. ChatGPT also opens up the possibility of building explanatory AI for stance detection.
翻译:立场检测是指从给定文本中提取对特定目标的态度(支持、反对或中立)的任务。随着社交媒体内容的激增,此类研究日益受到关注。处理立场检测的传统框架是将其转化为文本分类任务。深度学习模型已取代基于规则的模型和传统机器学习模型来解决此类问题。当前深度神经网络面临两大挑战:社交媒体帖子标注数据不足与信息不充分,以及深度学习模型难以解释的特性。2022年11月30日,新型预训练语言模型ChatGPT发布。针对立场检测任务,我们的实验表明,ChatGPT在SemEval-2016和P-Stance等常用数据集上能达到最先进或相近的性能。同时,ChatGPT能为其预测提供解释——这是现有任何模型都无法实现的能力。对于其无法给出分类结果的情况,所提供的解释尤为有用。ChatGPT有潜力成为NLP领域立场检测任务的最佳AI模型,或至少改变该领域的研究范式。ChatGPT也为构建可解释的立场检测AI开辟了可能性。