While humans naturally develop theory of mind (ToM), the capability to understand other people's mental states and beliefs, state-of-the-art large language models (LLMs) underperform on simple ToM benchmarks. We posit that we can extend our understanding of LLMs' ToM abilities by evaluating key human ToM precursors -- perception inference and perception-to-belief inference -- in LLMs. We introduce two datasets, Percept-ToMi and Percept-FANToM, to evaluate these precursory inferences for ToM in LLMs by annotating characters' perceptions on ToMi and FANToM, respectively. Our evaluation of eight state-of-the-art LLMs reveals that the models generally perform well in perception inference while exhibiting limited capability in perception-to-belief inference (e.g., lack of inhibitory control). Based on these results, we present PercepToM, a novel ToM method leveraging LLMs' strong perception inference capability while supplementing their limited perception-to-belief inference. Experimental results demonstrate that PercepToM significantly enhances LLM's performance, especially in false belief scenarios.

翻译：人类天生具备心智理论（ToM）能力，即理解他人心理状态和信念的能力，而当前最先进的大型语言模型（LLM）在简单的心智理论基准测试中表现欠佳。我们认为，通过评估LLM中人类心智理论的关键前置能力——感知推理与感知到信念的推理——可以拓展对LLM心智理论能力的理解。我们引入了两个数据集Percept-ToMi和Percept-FANToM，分别通过对ToMi和FANToM中角色感知的标注，来评估LLM中心智理论的前置推理能力。对八个前沿大型语言模型的评估表明，这些模型在感知推理方面普遍表现良好，但在感知到信念的推理方面能力有限（例如缺乏抑制控制）。基于这些发现，我们提出了PercepToM——一种新颖的心智理论方法，该方法利用LLM强大的感知推理能力，同时弥补其感知到信念推理的不足。实验结果表明，PercepToM显著提升了LLM在心智理论任务中的表现，尤其在错误信念场景中效果突出。