While numerous works have assessed the generative performance of language models (LMs) on tasks requiring Theory of Mind reasoning, research into the models' internal representation of mental states remains limited. Recent work has used probing to demonstrate that LMs can represent beliefs of themselves and others. However, these claims are accompanied by limited evaluation, making it difficult to assess how mental state representations are affected by model design and training choices. We report an extensive benchmark with various LM types with different model sizes, fine-tuning approaches, and prompt designs to study the robustness of mental state representations and memorisation issues within the probes. Our results show that the quality of models' internal representations of the beliefs of others increases with model size and, more crucially, with fine-tuning. We are the first to study how prompt variations impact probing performance on theory of mind tasks. We demonstrate that models' representations are sensitive to prompt variations, even when such variations should be beneficial. Finally, we complement previous activation editing experiments on Theory of Mind tasks and show that it is possible to improve models' reasoning performance by steering their activations without the need to train any probe.

翻译：尽管已有大量研究评估了语言模型（LMs）在需要心理理论推理任务上的生成性能，但关于模型内部心理状态表征的研究仍然有限。近期工作通过探针技术证明，语言模型能够表征自身及他人的信念。然而，这些论断所依据的评估较为有限，难以判断心理状态表征如何受模型设计与训练选择的影响。我们构建了一个涵盖不同模型规模、微调方法及提示设计的多样化语言模型基准测试，以探究心理状态表征的鲁棒性及探针内部的记忆化问题。研究结果表明，模型对他人信念的内部表征质量随模型规模提升而增强，且更关键的是，通过微调可显著改善表征效果。我们首次系统研究了提示词变化如何影响心理理论任务中的探针性能，并证明即使是有益的提示变化也会导致模型表征的敏感性。最后，我们补充了先前心理理论任务的激活编辑实验，表明无需训练任何探针，仅通过引导模型激活即可提升其推理性能。