Large Language Models (LLMs) do not differentially represent numbers, which are pervasive in text. In contrast, neuroscience research has identified distinct neural representations for numbers and words. In this work, we investigate how well popular LLMs capture the magnitudes of numbers (e.g., that $4 < 5$) from a behavioral lens. Prior research on the representational capabilities of LLMs evaluates whether they show human-level performance, for instance, high overall accuracy on standard benchmarks. Here, we ask a different question, one inspired by cognitive science: How closely do the number representations of LLMscorrespond to those of human language users, who typically demonstrate the distance, size, and ratio effects? We depend on a linking hypothesis to map the similarities among the model embeddings of number words and digits to human response times. The results reveal surprisingly human-like representations across language models of different architectures, despite the absence of the neural circuitry that directly supports these representations in the human brain. This research shows the utility of understanding LLMs using behavioral benchmarks and points the way to future work on the number of representations of LLMs and their cognitive plausibility.

翻译：大型语言模型（LLMs）无法以差异化方式表示文本中普遍存在的数字。相比之下，神经科学研究已识别出大脑对数字和词汇具有不同的神经表征。本研究从行为学视角探究主流LLM如何捕捉数值大小（例如，4<5）。以往关于LLM表征能力的研究主要评估其是否达到人类水平的表现，例如在标准基准测试中展现的高准确率。本文则提出一个源于认知科学的不同问题：LLM的数字表征与人类语言用户（通常表现出距离效应、大小效应和比率效应）的对应程度如何？我们依赖一个连接假设，将模型对数字词和数字的嵌入相似性映射到人类反应时。结果表明，尽管缺乏人类大脑中直接支持这些表征的神经回路，不同架构的语言模型仍展现出惊人地类人化的表征。本研究揭示了利用行为基准理解LLM的实用价值，并为未来关于LLM数字表征及其认知合理性的研究指明了方向。