Neural networks drive the success of natural language processing. A fundamental property of natural languages is their compositional structure, allowing us to describe new meanings systematically. However, neural networks notoriously struggle with systematic generalization and do not necessarily benefit from a compositional structure in emergent communication simulations. Here, we test how neural networks compare to humans in learning and generalizing a new language. We do this by closely replicating an artificial language learning study (conducted originally with human participants) and evaluating the memorization and generalization capabilities of deep neural networks with respect to the degree of structure in the input language. Our results show striking similarities between humans and deep neural networks: More structured linguistic input leads to more systematic generalization and better convergence between humans and neural network agents and between different neural agents. We then replicate this structure bias found in humans and our recurrent neural networks with a Transformer-based large language model (GPT-3), showing a similar benefit for structured linguistic input regarding generalization systematicity and memorization errors. These findings show that the underlying structure of languages is crucial for systematic generalization. Due to the correlation between community size and linguistic structure in natural languages, our findings underscore the challenge of automated processing of low-resource languages. Nevertheless, the similarity between humans and machines opens new avenues for language evolution research.

翻译：神经网络驱动了自然语言处理的成功。自然语言的一个基本属性是其组合结构，使我们能够系统地描述新的含义。然而，神经网络在系统泛化方面表现不佳，且在涌现交流模拟中未必受益于组合结构。本研究通过紧密复现一项人工语言学习实验（最初以人类参与者进行），评估深度神经网络对输入语言结构程度的记忆与泛化能力，以此比较神经网络与人类在学习及泛化新语言时的表现差异。结果显示人类与深度神经网络存在显著相似性：更具结构性的语言输入能够带来更系统的泛化能力，并提升人类与神经网络智能体之间、以及不同神经智能体之间的收敛一致性。我们随后将这种在人类与循环神经网络中观察到的结构偏好，扩展至基于Transformer的大语言模型（GPT-3），发现其在泛化系统性与记忆错误方面同样受益于结构化语言输入。这些发现表明，语言的底层结构对系统性泛化至关重要。鉴于自然语言中社群规模与语言结构的相关性，本研究结果凸显了低资源语言自动化处理的挑战。然而，人类与机器之间的相似性为语言演化研究开辟了新路径。