The observed similarities in the behavior of humans and Large Language Models (LLMs) have prompted researchers to consider the potential of using LLMs as models of human cognition. However, several significant challenges must be addressed before LLMs can be legitimately regarded as cognitive models. For instance, LLMs are trained on far more data than humans typically encounter, and may have been directly trained on human data in specific cognitive tasks or aligned with human preferences. Consequently, the origins of these behavioral similarities are not well understood. In this paper, we propose a novel way to enhance the utility of LLMs as cognitive models. This approach involves (i) leveraging computationally equivalent tasks that both an LLM and a rational agent need to master for solving a cognitive problem and (ii) examining the specific task distributions required for an LLM to exhibit human-like behaviors. We apply this approach to decision-making -- specifically risky and intertemporal choice -- where the key computationally equivalent task is the arithmetic of expected value calculations. We show that an LLM pretrained on an ecologically valid arithmetic dataset, which we call Arithmetic-GPT, predicts human behavior better than many traditional cognitive models. Pretraining LLMs on ecologically valid arithmetic datasets is sufficient to produce a strong correspondence between these models and human decision-making. Our results also suggest that LLMs used as cognitive models should be carefully investigated via ablation studies of the pretraining data.

翻译：人类行为与大型语言模型（LLM）之间观察到的相似性，促使研究者开始探讨将LLM用作人类认知模型的潜力。然而，在将LLM合法地视为认知模型之前，仍需解决若干重大挑战。例如，LLM的训练数据量远超人类通常接触的范围，且可能已在特定认知任务中直接使用人类数据进行训练，或与人类偏好进行了对齐。因此，这些行为相似性的根源尚未得到充分理解。本文提出一种增强LLM作为认知模型效用的新方法，该方法包括：（一）利用LLM与理性智能体在解决认知问题时均需掌握的计算等价任务；（二）探究使LLM展现类人行为所需的具体任务分布。我们将此方法应用于决策研究——特别是风险决策与跨期决策——其关键计算等价任务为期望值计算的算术运算。研究表明，在生态效度良好的算术数据集（我们称之为Arithmetic-GPT）上预训练的LLM，比许多传统认知模型更能准确预测人类行为。在生态有效的算术数据集上进行预训练，足以使这些模型与人类决策行为产生高度对应。我们的结果还表明，用作认知模型的LLM应通过预训练数据的消融实验进行审慎验证。