Large Language Models (LLMs) have recently showcased their remarkable capacities, not only in natural language processing tasks but also across diverse domains such as clinical medicine, legal consultation, and education. LLMs become more than mere applications, evolving into assistants capable of addressing diverse user requests. This narrows the distinction between human beings and artificial intelligence agents, raising intriguing questions regarding the potential manifestation of personalities, temperaments, and emotions within LLMs. In this paper, we propose a framework, PsychoBench, for evaluating diverse psychological aspects of LLMs. Comprising thirteen scales commonly used in clinical psychology, PsychoBench further classifies these scales into four distinct categories: personality traits, interpersonal relationships, motivational tests, and emotional abilities. Our study examines five popular models, namely text-davinci-003, gpt-3.5-turbo, gpt-4, LLaMA-2-7b, and LLaMA-2-13b. Additionally, we employ a jailbreak approach to bypass the safety alignment protocols and test the intrinsic natures of LLMs. We have made PsychoBench openly accessible via https://github.com/CUHK-ARISE/PsychoBench.

翻译：大语言模型（LLMs）近来展现了卓越的能力，不仅在自然语言处理任务中表现出色，还广泛应用于临床医学、法律咨询和教育等多个领域。大语言模型已超越单纯的应用范畴，发展为能够应对多样化用户需求的助手。这缩小了人类与人工智能体之间的差异，引发了一个引人入胜的问题：大语言模型中是否可能展现出人格、气质和情感等心理特征。本文提出了一个名为PsychoBench的框架，用于评估大语言模型的多维度心理特征。该框架包含临床心理学中常用的十三个量表，并将其进一步划分为四类：人格特质、人际关系、动机测试和情感能力。本研究考察了五个流行模型：text-davinci-003、gpt-3.5-turbo、gpt-4、LLaMA-2-7b和LLaMA-2-13b。此外，我们还采用了一种越狱方法来绕过安全对齐协议，以测试大语言模型的内在本质。PsychoBench已通过https://github.com/CUHK-ARISE/PsychoBench 公开开放访问。