Over the years, many researchers have seemingly made the same observation: Brain and language model activations exhibit some structural similarities, enabling linear partial mappings between features extracted from neural recordings and computational language models. In an attempt to evaluate how much evidence has been accumulated for this observation, we survey over 30 studies spanning 10 datasets and 8 metrics. How much evidence has been accumulated, and what, if anything, is missing before we can draw conclusions? Our analysis of the evaluation methods used in the literature reveals that some of the metrics are less conservative. We also find that the accumulated evidence, for now, remains ambiguous, but correlations with model size and quality provide grounds for cautious optimism.

翻译：多年来，许多研究者似乎观察到同一个现象：大脑与语言模型激活状态在结构上存在某些相似性，使得从神经记录中提取的特征与计算语言模型之间能够建立线性部分映射。为评估这一观察已积累多少证据，我们综述了涵盖10个数据集和8项指标的30余项研究。当前已积累了多少证据？在得出明确结论前，是否存在尚未填补的空白？通过对文献中评估方法的分析，我们发现某些指标不够保守。同时，目前积累的证据仍显模糊，但模型规模与质量的相关性为谨慎乐观提供了依据。