Are large language models (LLMs) sensitive to the distinction between humanly possible and impossible languages? This question was recently used in a broader debate on whether LLMs and humans share the same innate learning biases. Previous work has answered it in the positive by comparing LLM learning curves on existing language datasets and on "impossible" datasets derived from them via various perturbation functions. Using the same methodology, we examine this claim on a wider set of languages and impossible perturbations. We find that in most cases, GPT-2 learns each language and its impossible counterpart equally easily, in contrast to previous findings. We also apply a more lenient condition by testing whether GPT-2 provides any kind of separation between the whole sets of natural vs. impossible languages, based on cross-linguistic variance in metrics derived from the learning curves. Taken together, these perspectives show that GPT-2 provides no systematic separation between the possible and the impossible.

翻译：大型语言模型（LLMs）是否对人类可能语言与不可能语言之间的区分敏感？这一问题近期被用于更广泛的争论，探讨LLMs与人类是否共享相同的先天学习偏差。已有研究通过比较LLM在现有语言数据集及其通过多种扰动函数派生的"不可能"数据集上的学习曲线，给出了肯定答案。采用相同方法，我们在更广泛的语言和不可能扰动上检验了这一论断。研究发现，在大多数情况下，GPT-2对每种语言及其不可能对应物的学习难度相当，这与先前结论相悖。我们还应用了更宽松的条件，基于学习曲线指标的跨语言变异，测试GPT-2是否在自然语言与不可能语言的整体集合间产生任何区分。综合来看，这些视角表明GPT-2未能系统区分可能语言与不可能语言。