In this paper, we classify scientific articles in the domain of natural language processing (NLP) and machine learning (ML), as core subfields of artificial intelligence (AI), into whether (i) they extend the current state-of-the-art by the introduction of novel techniques which beat existing models or whether (ii) they mainly criticize the existing state-of-the-art, i.e. that it is deficient with respect to some property (e.g. wrong evaluation, wrong datasets, misleading task specification). We refer to contributions under (i) as having a 'positive stance' and contributions under (ii) as having a 'negative stance' (to related work). We annotate over 1.5 k papers from NLP and ML to train a SciBERT-based model to automatically predict the stance of a paper based on its title and abstract. We then analyse large-scale trends on over 41 k papers from the last approximately 35 years in NLP and ML, finding that papers have become substantially more positive over time, but negative papers also got more negative and we observe considerably more negative papers in recent years. Negative papers are also more influential in terms of citations they receive.

翻译：本文对自然语言处理（NLP）和机器学习（ML）这两个人工智能（AI）核心子领域的科学论文进行分类，以判断其是否（i）通过引入新技术超越现有模型，从而拓展当前最优方法；或（ii）主要批判现有最优方法，即指其在某些属性上存在不足（例如评估错误、数据集不当、任务定义具有误导性）。我们将（i）类贡献称为“正面立场”，（ii）类贡献称为“负面立场”（针对相关工作）。我们标注了NLP和ML领域的1500余篇论文，基于SciBERT模型训练自动预测工具，使其能够根据论文标题和摘要判断其立场。随后，我们分析了过去约35年间NLP和ML领域超过4.1万篇论文的大规模趋势，发现论文的正面性随时间显著增强，但负面论文的批判力度亦有所加剧，且近年来负面论文数量明显增加。此外，负面论文在引用影响力方面表现更突出。