Neural language models, particularly large-scale ones, have been consistently proven to be most effective in predicting brain neural activity across a range of studies. However, previous research overlooked the comparison of these models with psychologically plausible ones. Moreover, evaluations were reliant on limited, single-modality, and English cognitive datasets. To address these questions, we conducted an analysis comparing encoding performance of various neural language models and psychologically plausible models. Our study utilized extensive multi-modal cognitive datasets, examining bilingual word and discourse levels. Surprisingly, our findings revealed that psychologically plausible models outperformed neural language models across diverse contexts, encompassing different modalities such as fMRI and eye-tracking, and spanning languages from English to Chinese. Among psychologically plausible models, the one incorporating embodied information emerged as particularly exceptional. This model demonstrated superior performance at both word and discourse levels, exhibiting robust prediction of brain activation across numerous regions in both English and Chinese.

翻译：神经语言模型，尤其是大规模模型，已在多项研究中被一致证明能最有效地预测大脑神经活动。然而，以往研究忽略了这些模型与心理合理模型的比较。此外，评估依赖于有限的、单模态的英语认知数据集。为解决这些问题，我们进行了分析，比较了多种神经语言模型与心理合理模型的编码性能。本研究利用了广泛的多模态认知数据集，考察了双语词汇和语篇层面。令人惊讶的是，我们的研究发现，心理合理模型在多种情境下——涵盖不同模态（如fMRI和眼动追踪）以及从英语到汉语的语言——均优于神经语言模型。在心理合理模型中，包含具身信息的模型表现尤为突出。该模型在词汇和语篇层面均展现出优越性能，能够稳健预测英语和汉语中多个脑区的脑激活模式。