The advent of large language models (LLMs) has enabled significant performance gains in the field of natural language processing. However, recent studies have found that LLMs often resort to shortcuts when performing tasks, creating an illusion of enhanced performance while lacking generalizability in their decision rules. This phenomenon introduces challenges in accurately assessing natural language understanding in LLMs. Our paper provides a concise survey of relevant research in this area and puts forth a perspective on the implications of shortcut learning in the evaluation of language models, specifically for NLU tasks. This paper urges more research efforts to be put towards deepening our comprehension of shortcut learning, contributing to the development of more robust language models, and raising the standards of NLU evaluation in real-world scenarios.

翻译：大语言模型的出现使得自然语言处理领域取得了显著的性能提升。然而，近期研究发现，大语言模型在执行任务时常常采取捷径，这造成了性能提升的假象，但其决策规则缺乏泛化能力。这一现象给准确评估大语言模型的自然语言理解能力带来了挑战。本文对该领域的相关研究进行了简要综述，并就捷径学习对语言模型评估（特别是自然语言理解任务）的影响提出了见解。本文呼吁投入更多研究工作，以加深对捷径学习的理解，助力开发更鲁棒的语言模型，并提升现实场景中自然语言理解评估的标准。