Large language models (LLMs) are currently at the forefront of intertwining AI systems with human communication and everyday life. Due to rapid technological advances and their extreme versatility, LLMs nowadays have millions of users and are at the cusp of being the main go-to technology for information retrieval, content generation, problem-solving, etc. Therefore, it is of great importance to thoroughly assess and scrutinize their capabilities. Due to increasingly complex and novel behavioral patterns in current LLMs, this can be done by treating them as participants in psychology experiments that were originally designed to test humans. For this purpose, the paper introduces a new field of research called "machine psychology". The paper outlines how different subfields of psychology can inform behavioral tests for LLMs. It defines methodological standards for machine psychology research, especially by focusing on policies for prompt designs. Additionally, it describes how behavioral patterns discovered in LLMs are to be interpreted. In sum, machine psychology aims to discover emergent abilities in LLMs that cannot be detected by most traditional natural language processing benchmarks.

翻译：大型语言模型（LLMs）目前处于人工智能系统与人类沟通及日常生活交织的前沿。由于技术的快速进步及其极强的通用性，LLMs如今拥有数百万用户，并即将成为信息检索、内容生成、问题求解等领域的主要技术。因此，全面评估和审视它们的能力至关重要。由于当前LLMs展现出日益复杂和新颖的行为模式，一种有效的方法是将它们视为心理学实验中的受试者——这些实验原本是为测试人类而设计的。为此，本文提出一个名为“机器心理学”的新研究领域。本文阐述了心理学的不同分支如何为LLMs的行为测试提供参考，并界定了机器心理学研究的方法论标准，尤其侧重于提示设计的策略。此外，本文还描述了如何解读在LLMs中发现的行为模式。总之，机器心理学旨在揭示那些大多数传统自然语言处理基准无法检测到的LLMs新兴能力。