Deep Language Models (DLMs) provide a novel computational paradigm for understanding the mechanisms of natural language processing in the human brain. Unlike traditional psycholinguistic models, DLMs use layered sequences of continuous numerical vectors to represent words and context, allowing a plethora of emerging applications such as human-like text generation. In this paper we show evidence that the layered hierarchy of DLMs may be used to model the temporal dynamics of language comprehension in the brain by demonstrating a strong correlation between DLM layer depth and the time at which layers are most predictive of the human brain. Our ability to temporally resolve individual layers benefits from our use of electrocorticography (ECoG) data, which has a much higher temporal resolution than noninvasive methods like fMRI. Using ECoG, we record neural activity from participants listening to a 30-minute narrative while also feeding the same narrative to a high-performing DLM (GPT2-XL). We then extract contextual embeddings from the different layers of the DLM and use linear encoding models to predict neural activity. We first focus on the Inferior Frontal Gyrus (IFG, or Broca's area) and then extend our model to track the increasing temporal receptive window along the linguistic processing hierarchy from auditory to syntactic and semantic areas. Our results reveal a connection between human language processing and DLMs, with the DLM's layer-by-layer accumulation of contextual information mirroring the timing of neural activity in high-order language areas.

翻译：深度语言模型（DLMs）为理解人脑自然语言处理机制提供了新的计算范式。与传统心理语言学模型不同，DLMs 使用连续数值向量的分层序列来表示词汇和上下文，从而推动了类人文本生成等众多新兴应用。本文通过证明DLM层深度与各层预测人脑活动的时间点之间的强相关性，揭示DLM的层级层次可用于建模大脑语言理解的时间动态。我们之所以能够实现单层时间解析，得益于采用具有比fMRI等无创方法更高时间分辨率的皮层脑电图（ECoG）数据。实验通过ECoG记录受试者聆听30分钟叙事时的神经活动，同时将同一叙事输入高性能DLM（GPT2-XL）。接着，我们从DLM不同层提取上下文嵌入，并利用线性编码模型预测神经活动。研究首先聚焦于额下回（IFG，即布罗卡区），随后扩展模型以追踪从听觉区域到句法和语义区域的言语处理层级中逐渐扩大的时间接收野。我们的结果揭示了人类语言处理与DLM之间的联系：DLM逐层积累上下文信息的方式，与高级语言区域中神经活动的时间特征相呼应。