We introduce SPOT (Stopping Points in Online Threads), the first annotated corpus translating the sociological concept of stopping point into a reproducible NLP task. Stopping points are ordinary critical interventions that pause or redirect online discussions through a range of forms (irony, subtle doubt or fragmentary arguments) that frameworks like counterspeech or social correction often overlook. We operationalize this concept as a binary classification task and provide reliable annotation guidelines. The corpus contains 43,305 manually annotated French Facebook comments linked to URLs flagged as false information by social media users, enriched with contextual metadata (article, post, parent comment, page or group, and source). We benchmark fine-tuned encoder models (CamemBERT) and instruction-tuned LLMs under various prompting strategies. Results show that fine-tuned encoders outperform prompted LLMs in F1 score by more than 10 percentage points, confirming the importance of supervised learning for emerging non-English social media tasks. Incorporating contextual metadata further improves encoder models F1 scores from 0.75 to 0.78. We release the anonymized dataset, along with the annotation guidelines and code in our code repository, to foster transparency and reproducible research.
翻译:本文介绍SPOT(在线线程中的停止点),这是首个将社会学概念“停止点”转化为可复现自然语言处理任务的标注语料库。停止点指通过一系列形式(反讽、微妙质疑或碎片化论证)来暂停或扭转在线讨论的普通关键干预,此类形式常被反制言论或社会矫正等框架所忽视。我们将此概念操作化为二元分类任务,并提供可靠的标注指南。该语料库包含43,305条经人工标注的法语Facebook评论,这些评论均关联至社交媒体用户标记为虚假信息的URL,并附有上下文元数据(文章、帖子、父级评论、页面/群组及信息来源)。我们在多种提示策略下对微调编码器模型(CamemBERT)和指令调优大语言模型进行了基准测试。结果表明,微调编码器在F1分数上比提示大语言模型高出10个百分点以上,证实了监督学习对于新兴非英语社交媒体任务的重要性。引入上下文元数据后,编码器模型的F1分数从0.75进一步提升至0.78。为促进研究透明度和可复现性,我们公开了匿名化数据集、标注指南及代码仓库。