Neural networks drive the success of natural language processing. A fundamental property of language is its compositional structure, allowing humans to produce forms for new meanings systematically. However, unlike humans, neural networks notoriously struggle with systematic generalization, and do not necessarily benefit from compositional structure in emergent communication simulations. This poses a problem for using neural networks to simulate human language learning and evolution, and suggests crucial differences in the biases of the different learning systems. Here, we directly test how neural networks compare to humans in learning and generalizing different input languages that vary in their degree of structure. We evaluate the memorization and generalization capabilities of a pre-trained language model GPT-3.5 (analagous to an adult second language learner) and recurrent neural networks trained from scratch (analaogous to a child first language learner). Our results show striking similarities between deep neural networks and adult human learners, with more structured linguistic input leading to more systematic generalization and to better convergence between neural networks and humans. These findings suggest that all the learning systems are sensitive to the structure of languages in similar ways with compositionality being advantageous for learning. Our findings draw a clear prediction regarding children's learning biases, as well as highlight the challenges of automated processing of languages spoken by small communities. Notably, the similarity between humans and machines opens new avenues for research on language learning and evolution.

翻译：神经网络推动了自然语言处理领域的成功。语言的一个基本属性是其组合结构，这使得人类能够系统地生成新意义对应的形式。然而，与人类不同，神经网络在系统泛化方面表现欠佳，且在涌现通信模拟中并不必然受益于组合结构。这给利用神经网络模拟人类语言学习与演化带来了难题，并揭示了不同学习系统在偏好上的关键差异。本文直接测试了神经网络与人类在学习并泛化不同结构程度输入语言时的表现对比。我们评估了预训练语言模型GPT-3.5（类比成人第二语言学习者）的 memorize 与泛化能力，以及从零训练的循环神经网络（类比儿童第一语言学习者）的表现。结果显示，深度神经网络与成人人类学习者之间存在显著相似性：更具结构性的语言输入导致更系统的泛化，并促进神经网络与人类之间的更好趋同。这些发现表明，所有学习系统均以类似方式对语言结构敏感，其中组合性对学习有利。我们的研究结果为儿童学习偏好提供了明确预测，同时凸显了自动处理小社区语言的挑战。值得注意的是，人类与机器之间的相似性为语言学习与演化的研究开辟了新途径。