Large language models (LLMs) have complicated internal dynamics, but induce representations of words and phrases whose geometry we can study. Human language processing is also opaque, but neural response measurements can provide (noisy) recordings of activation during listening or reading, from which we can extract similar representations of words and phrases. Here we study the extent to which the geometries induced by these representations, share similarities in the context of brain decoding. We find that the larger neural language models get, the more their representations are structurally similar to neural response measurements from brain imaging. Code is available at \url{https://github.com/coastalcph/brainlm}.

翻译：大型语言模型（LLMs）具有复杂的内部动态机制，但其所诱导的词和短语表征的几何结构可供研究。人类语言处理过程同样难以直接观测，但通过神经响应测量，我们能够获取听读过程中（含噪声的）激活记录，并从中提取类似的词与短语表征。本研究探究了在脑解码背景下，这些表征所引发的几何结构在多大程度上具有相似性。我们发现，神经语言模型的规模越大，其表征与脑成像中的神经响应测量结果在结构上越相似。相关代码已开源在 \url{https://github.com/coastalcph/brainlm}。