Advances in computational methods and big data availability have recently translated into breakthroughs in AI applications. With successes in bottom-up challenges partially overshadowing shortcomings, the 'human-like' performance of Large Language Models has raised the question of how linguistic performance is achieved by algorithms. Given systematic shortcomings in generalization across many AI systems, in this work we ask whether linguistic performance is indeed guided by language knowledge in Large Language Models. To this end, we prompt GPT-3 with a grammaticality judgement task and comprehension questions on less frequent constructions that are thus unlikely to form part of Large Language Models' training data. These included grammatical 'illusions', semantic anomalies, complex nested hierarchies and self-embeddings. GPT-3 failed for every prompt but one, often offering answers that show a critical lack of understanding even of high-frequency words used in these less frequent grammatical constructions. The present work sheds light on the boundaries of the alleged AI human-like linguistic competence and argues that, far from human-like, the next-word prediction abilities of LLMs may face issues of robustness, when pushed beyond training data.

翻译：计算方法的进步和大数据的可用性最近转化为人工智能应用的突破。随着自下而上挑战的成功在一定程度上掩盖了不足之处，大型语言模型的“类人”表现引发了关于算法如何实现语言能力的问题。鉴于许多人工智能系统在泛化方面存在系统性缺陷，本研究探究了大型语言模型的语言表现是否确实由语言知识引导。为此，我们使用GPT-3进行语法性判断任务和关于低频结构的理解问题，这些结构不太可能构成大型语言模型训练数据的一部分。这些结构包括语法“错觉”、语义异常、复杂的嵌套层级和自嵌入结构。除一个提示外，GPT-3在所有提示中均失败，常常给出的答案甚至对这些低频语法结构中使用的高频词表现出严重缺乏理解。本研究揭示了所谓的人工智能类人语言能力的边界，并认为，远非类人，大型语言模型的下一个词预测能力在超出训练数据时可能面临稳健性问题。