In Chomsky's provocative critique "The False Promise of CHATGPT," Large Language Models (LLMs) are characterized as mere pattern predictors that do not acquire languages via intrinsic causal and self-correction structures like humans, therefore are not able to distinguish impossible languages. It stands as a representative in a fundamental challenge to the intellectual foundations of AI, for it integrally synthesizes major issues in methodologies within LLMs and possesses an iconic a priori rationalist perspective. We examine this famous critic from both the perspective in pre-existing literature of linguistics and psychology as well as a research based on an experiment inquiring the capacity of learning both possible and impossible languages among LLMs. We constructed a set of syntactically impossible languages by applying certain transformations to English. These include reversing whole sentences, and adding negation based on word-count parity. Two rounds of controlled experiments were each conducted on GPT-2 small models and long short-term memory (LSTM) models. Statistical analysis (Welch's t-test) shows GPT2 small models underperform in learning all of the impossible languages compared to their performance on the possible language (p<.001). On the other hand, LSTM models' performance tallies with Chomsky's argument, suggesting the irreplaceable role of the evolution of transformer architecture. Based on theoretical analysis and empirical findings, we propose a new vision within Chomsky's theory towards LLMs, and a shift of theoretical paradigm outside Chomsky, from his "rationalist-romantics" paradigm to functionalism and empiricism in LLMs research.

翻译：在乔姆斯基的挑衅性批评《CHATGPT的虚假承诺》中，大语言模型（LLMs）被描述为仅仅是模式预测器，无法像人类那样通过内在的因果和自我修正结构来习得语言，因此不能区分不可能语言。这一批评对人工智能的智力基础构成了根本性挑战，因为它综合了大语言模型方法论中的主要问题，并具有标志性的先验理性主义视角。我们从语言学与心理学既有文献的视角，以及一项探究大语言模型学习可能语言与不可能语言能力的实验研究出发，审视了这一著名批评。我们通过对英语施加特定转换，构建了一组句法上不可能的语言。这些转换包括完全反转句子，以及基于单词计数的奇偶性添加否定。我们在GPT-2小型模型和长短期记忆（LSTM）模型上分别进行了两轮对照实验。统计分析（韦尔奇t检验）显示，与在可能语言上的表现相比，GPT-2小型模型在学习所有不可能语言时表现不佳（p<.001）。另一方面，LSTM模型的表现与乔姆斯基的论点相符，这暗示了Transformer架构演变的不可替代作用。基于理论分析和实证发现，我们提出了在乔姆斯基理论框架内看待大语言模型的新视角，以及在乔姆斯基框架之外的理论范式转变——从其"理性主义-浪漫主义"范式转向大语言模型研究中的功能主义和经验主义。