Language models (LMs), despite their advances, often depend on spurious correlations, undermining their accuracy and generalizability. This study addresses the overlooked impact of subtler, more complex shortcuts that compromise model reliability beyond oversimplified shortcuts. We introduce a comprehensive benchmark that categorizes shortcuts into occurrence, style, and concept, aiming to explore the nuanced ways in which these shortcuts influence the performance of LMs. Through extensive experiments across traditional LMs, large language models, and state-of-the-art robust models, our research systematically investigates models' resilience and susceptibilities to sophisticated shortcuts. Our benchmark and code can be found at: https://github.com/yuqing-zhou/shortcut-learning-in-text-classification.

翻译：尽管语言模型取得了进展，但其往往依赖于伪相关性，这削弱了模型的准确性和泛化能力。本研究探讨了被忽视的、更微妙复杂的捷径所产生的影响，这些捷径超越了过度简化的捷径，进一步损害了模型的可靠性。我们引入了一个综合性基准，将捷径分为出现型、风格型和概念型三类，旨在探究这些捷径影响语言模型性能的细微方式。通过对传统语言模型、大语言模型以及最先进的鲁棒模型进行大量实验，我们的研究系统性地考察了模型对复杂捷径的抵抗力和易感性。我们的基准和代码可在以下网址找到：https://github.com/yuqing-zhou/shortcut-learning-in-text-classification。