Humans represent scenes and objects in rich feature spaces, carrying information that allows us to generalise about category memberships and abstract functions with few examples. What determines whether a neural network model generalises like a human? We tested how well the representations of $86$ pretrained neural network models mapped to human learning trajectories across two tasks where humans had to learn continuous relationships and categories of natural images. In these tasks, both human participants and neural networks successfully identified the relevant stimulus features within a few trials, demonstrating effective generalisation. We found that while training dataset size was a core determinant of alignment with human choices, contrastive training with multi-modal data (text and imagery) was a common feature of currently publicly available models that predicted human generalisation. Intrinsic dimensionality of representations had different effects on alignment for different model types. Lastly, we tested three sets of human-aligned representations and found no consistent improvements in predictive accuracy compared to the baselines. In conclusion, pretrained neural networks can serve to extract representations for cognitive models, as they appear to capture some fundamental aspects of cognition that are transferable across tasks. Both our paradigms and modelling approach offer a novel way to quantify alignment between neural networks and humans and extend cognitive science into more naturalistic domains.

翻译：人类在丰富的特征空间中表征场景和物体，这些特征携带的信息使我们能够通过少量示例对类别成员关系和抽象功能进行泛化。是什么决定了神经网络模型能否像人类一样泛化？我们测试了86个预训练神经网络模型的表征在两个任务中与人类学习轨迹的映射程度，在这两个任务中，人类需要学习自然图像的连续关系和类别。在这些任务中，人类参与者和神经网络都能在少量试次内识别出相关的刺激特征，展现出有效的泛化能力。我们发现，虽然训练数据集大小是与人类选择对齐的核心决定因素，但当前可公开获取的模型中，采用多模态数据（文本和图像）进行对比训练是预测人类泛化能力的共同特征。表征的内在维度对不同模型类型的对齐性产生了不同影响。最后，我们测试了三组与人类对齐的表征，发现与基线相比，预测准确性并未获得一致提升。总之，预训练神经网络可用于提取认知模型的表征，因为它们似乎捕捉了跨任务可迁移的认知基本方面。我们的实验范式和建模方法为量化神经网络与人类之间的对齐性提供了一种新途径，并将认知科学拓展至更自然的领域。