In Chomsky's provocative critique "The False Promise of CHATGPT," Large Language Models (LLMs) are characterized as mere pattern predictors that do not acquire languages via intrinsic causal and self-correction structures like humans, therefore are not able to distinguish impossible languages. It stands as a representative in a fundamental challenge to the intellectual foundations of AI, for it integrally synthesizes major issues in methodologies within LLMs and possesses an iconic a priori rationalist perspective. We examine this famous critic from both the perspective in pre-existing literature of linguistics and psychology as well as a research based on an experiment inquiring the capacity of learning both possible and impossible languages among LLMs. We constructed a set of syntactically impossible languages by applying certain transformations to English. These include reversing whole sentences, and adding negation based on word-count parity. Two rounds of controlled experiments were each conducted on GPT-2 small models and long short-term memory (LSTM) models. Statistical analysis (Welch's t-test) shows GPT2 small models underperform in learning all of the impossible languages compared to their performance on the possible language (p<.001). On the other hand, LSTM models' performance tallies with Chomsky's argument, suggesting the irreplaceable role of the evolution of transformer architecture. Based on theoretical analysis and empirical findings, we propose a new vision within Chomsky's theory towards LLMs, and a shift of theoretical paradigm outside Chomsky, from his "rationalist-romantics" paradigm to functionalism and empiricism in LLMs research.

翻译：在乔姆斯基的挑衅性批判《ChatGPT的虚假承诺》中，大型语言模型被描述为仅仅是模式预测器，无法像人类那样通过内在因果和自我修正结构习得语言，因此不能区分不可能语言。这一批判构成了对人工智能理论基础的根本性质疑，因其整合了大型语言模型方法论中的主要问题，并具有标志性的先验理性主义视角。我们从语言学与心理学既有文献视角出发，结合一项探究大型语言模型学习可能及不可能语言能力的实验研究，对这一著名批判进行了审视。我们通过对英语施加特定变换，构建了一组句法上不可能的语言。这些变换包括整句反转，以及基于单词计数字偶性的否定添加。分别在GPT-2小型模型和长短期记忆模型上进行了两轮对照实验。统计分析显示，与可能语言的学习表现相比，GPT-2小型模型在所有不可能语言的学习中均表现欠佳。另一方面，LSTM模型的表现与乔姆斯基的论点相符，这暗示了Transformer架构演进不可替代的作用。基于理论分析和实证发现，我们提出了乔姆斯基理论框架内审视大型语言模型的新视角，以及超越乔姆斯基范式的理论转向——从其"理性主义-浪漫主义"范式转向大型语言模型研究中的功能主义与经验主义。