Large Language Models (LLMs) are increasingly used in everyday life and research. One of the most common use cases is conversational interactions, enabled by the language generation capabilities of LLMs. Just as between two humans, a conversation between an LLM-powered entity and a human depends on the personality of the conversants. However, measuring the personality of a given LLM is currently a challenge. This paper introduces the Language Model Linguistic Personality Assessment (LMLPA), a system designed to evaluate the linguistic personalities of LLMs. Our system helps to understand LLMs' language generation capabilities by quantitatively assessing the distinct personality traits reflected in their linguistic outputs. Unlike traditional human-centric psychometrics, the LMLPA adapts a personality assessment questionnaire, specifically the Big Five Inventory, to align with the operational capabilities of LLMs, and also incorporates the findings from previous language-based personality measurement literature. To mitigate sensitivity to the order of options, our questionnaire is designed to be open-ended, resulting in textual answers. Thus, the AI rater is needed to transform ambiguous personality information from text responses into clear numerical indicators of personality traits. Utilising Principal Component Analysis and reliability validations, our findings demonstrate that LLMs possess distinct personality traits that can be effectively quantified by the LMLPA. This research contributes to Human-Computer Interaction and Human-Centered AI, providing a robust framework for future studies to refine AI personality assessments and expand their applications in multiple areas, including education and manufacturing.

翻译：大型语言模型（LLM）在日常生活中的应用日益广泛。其中最常见的应用场景之一是利用LLM的语言生成能力实现对话交互。正如人与人之间的对话一样，由LLM驱动的实体与人类之间的对话也取决于对话者的个性特征。然而，目前评估特定LLM的个性仍面临挑战。本文提出了语言模型语言个性评估系统（LMLPA），该系统旨在评估LLM的语言个性。通过定量分析LLM语言输出中反映的独特个性特征，我们的系统有助于深入理解LLM的语言生成能力。与传统以人为中心的心理测量方法不同，LMLPA对个性评估问卷（特别是大五人格量表）进行了适应性改造，使其与LLM的运算能力相匹配，并整合了先前基于语言的个性测量文献的研究成果。为降低对选项顺序的敏感性，我们设计的问卷采用开放式问题形式，从而获得文本答案。因此需要借助AI评估器将文本回答中模糊的个性信息转化为清晰的人格特质数值指标。通过主成分分析和信效度验证，我们的研究结果表明LLM具有可通过LMLPA有效量化的独特个性特征。这项研究为人机交互和以人为中心的人工智能领域作出贡献，为未来研究提供了完善AI个性评估的稳健框架，并拓展了其在教育和制造等多个领域的应用前景。