Do machines and humans process language in similar ways? Recent research has hinted in the affirmative, finding that brain signals can be effectively predicted using the internal representations of language models (LMs). Although such results are thought to reflect shared computational principles between LMs and human brains, there are also clear differences in how LMs and humans represent and use language. In this work, we systematically explore the divergences between human and machine language processing by examining the differences between LM representations and human brain responses to language as measured by Magnetoencephalography (MEG) across two datasets in which subjects read and listened to narrative stories. Using a data-driven approach, we identify two domains that are not captured well by LMs: social/emotional intelligence and physical commonsense. We then validate these domains with human behavioral experiments and show that fine-tuning LMs on these domains can improve their alignment with human brain responses.

翻译：机器和人类以相似的方式处理语言吗？近期研究给出了肯定答案，发现语言模型（LM）的内部表征能够有效预测脑信号。尽管这类结果被认为反映了语言模型与人脑之间存在共享的计算原理，但两者在语言表征与使用方式上仍存在明显差异。本研究通过考察语言模型表征与人脑对语言的反应（通过脑磁图（MEG）测量）之间的差异，系统探索了人类与机器语言处理的分歧点。研究基于两个数据集（受试者阅读和聆听叙事性故事），采用数据驱动方法，识别出语言模型难以很好捕获的两个领域：社会/情感智能以及物理常识。我们随后通过人类行为实验验证了这些领域，并证明在这些领域对语言模型进行微调能够提升其与人脑反应的匹配度。