Negation is a common everyday phenomena and has been a consistent area of weakness for language models (LMs). Although the Information Retrieval (IR) community has adopted LMs as the backbone of modern IR architectures, there has been little to no research in understanding how negation impacts neural IR. We therefore construct a straightforward benchmark on this theme: asking IR models to rank two documents that differ only by negation. We show that the results vary widely according to the type of IR architecture: cross-encoders perform best, followed by late-interaction models, and in last place are bi-encoder and sparse neural architectures. We find that most current information retrieval models do not consider negation, performing similarly or worse than randomly ranking. We show that although the obvious approach of continued fine-tuning on a dataset of contrastive documents containing negations increases performance (as does model size), there is still a large gap between machine and human performance.

翻译：否定是一种常见的日常语言现象，且一直是语言模型的薄弱环节。尽管信息检索领域已将语言模型作为现代IR架构的核心，但关于否定如何影响神经信息检索的研究几乎空白。为此，我们构建了一个关于该主题的直观基准测试：要求IR模型对两个仅因否定而不同的文档进行排序。结果表明，不同IR架构的表现差异显著：交叉编码器表现最佳，其次是后期交互模型，双编码器和稀疏神经架构表现最差。我们发现，大多数现有信息检索模型未考虑否定机制，其排序效果与随机排序相当甚至更差。研究还表明，尽管在包含否定的对比文档数据集上持续微调这一直观方法（以及扩大模型规模）可以提升性能，但机器与人类表现之间仍存在巨大差距。