Large Language Models (LLMs) do not differentially represent numbers, which are pervasive in text. In contrast, neuroscience research has identified distinct neural representations for numbers and words. In this work, we investigate how well popular LLMs capture the magnitudes of numbers (e.g., that $4 < 5$) from a behavioral lens. Prior research on the representational capabilities of LLMs evaluates whether they show human-level performance, for instance, high overall accuracy on standard benchmarks. Here, we ask a different question, one inspired by cognitive science: How closely do the number representations of LLMscorrespond to those of human language users, who typically demonstrate the distance, size, and ratio effects? We depend on a linking hypothesis to map the similarities among the model embeddings of number words and digits to human response times. The results reveal surprisingly human-like representations across language models of different architectures, despite the absence of the neural circuitry that directly supports these representations in the human brain. This research shows the utility of understanding LLMs using behavioral benchmarks and points the way to future work on the number representations of LLMs and their cognitive plausibility.

翻译：大型语言模型(LLMs)无法差异化表征文本中普遍存在的数字。相反，神经科学研究已识别出数字与词语具有不同的神经表征。本研究从行为视角考察主流LLMs如何捕捉数值大小（例如4<5）。先前关于LLMs表征能力的研究评估其是否展现人类水平性能，例如在标准基准测试中达到较高整体准确率。而本文提出一个受认知科学启发的不同问题：LLMs的数字表征与人类语言使用者的表征在多大程度上对应？后者通常表现出距离效应、大小效应和比率效应。我们依赖联结假说，将数字词和数字的模型嵌入向量间的相似性映射为人类反应时间。结果表明，尽管缺乏人脑中直接支持这些表征的神经回路，不同架构的语言模型仍展现出惊人相似的人类化表征。本研究展示了使用行为基准理解LLMs的效用，并为未来研究LLMs的数字表征及其认知合理性指明了方向。