The use of neural language models to model human behavior has met with mixed success. While some work has found that the surprisal estimates from these models can be used to predict a wide range of human neural and behavioral responses, other work studying more complex syntactic phenomena has found that these surprisal estimates generate incorrect behavioral predictions. This paper explores the extent to which the misalignment between empirical and model-predicted behavior can be minimized by training models on more developmentally plausible data, such as in the BabyLM Challenge. We trained teacher language models on the BabyLM "strict-small" dataset and used sentence level surprisal estimates from these teacher models to create a curriculum. We found tentative evidence that our curriculum made it easier for models to acquire linguistic knowledge from the training data: on the subset of tasks in the BabyLM challenge suite evaluating models' grammatical knowledge of English, models first trained on the BabyLM data curriculum and then on a few randomly ordered training epochs performed slightly better than models trained on randomly ordered epochs alone. This improved linguistic knowledge acquisition did not result in better alignment with human reading behavior, however: models trained on the BabyLM dataset (with or without a curriculum) generated predictions that were as misaligned with human behavior as models trained on larger less curated datasets. This suggests that training on developmentally plausible datasets alone is likely insufficient to generate language models capable of accurately predicting human language processing.

翻译：使用神经语言模型模拟人类行为的研究结果好坏参半。虽然部分研究发现这些模型的惊奇度估计值可用于预测广泛的人类神经与行为反应，但另一些针对更复杂句法现象的研究表明，这些惊奇度估计值会产生错误的行为预测。本文探讨了通过训练模型使用更具发展合理性的数据（如BabyLM挑战赛中的数据）来最小化经验行为与模型预测行为之间错位的程度。我们基于BabyLM“严格-小型”数据集训练教师语言模型，并利用这些教师模型输出的句子级惊奇度估计值构建课程。初步证据表明，我们的课程使模型更易于从训练数据中获取语言知识：在BabyLM挑战赛评估模型英语语法知识的任务子集上，先接受BabyLM数据课程训练再进行若干随机顺序训练轮次的模型，其表现略优于仅接受随机顺序训练轮次的模型。然而，这种语言知识获取的改进并未带来与人类阅读行为的更好对齐：基于BabyLM数据集（无论是否使用课程）训练的模型产生的预测，与基于更大规模未精心筛选数据集训练的模型一样，与人类行为存在错位。这表明，仅凭训练于发展合理性数据集可能不足以生成能准确预测人类语言处理的语言模型。