Large Language Models (LLMs) are increasingly used in everyday life and research. One of the most common use cases is conversational interactions, enabled by the language generation capabilities of LLMs. Just as between two humans, a conversation between an LLM-powered entity and a human depends on the personality of the conversants. However, measuring the personality of a given LLM is currently a challenge. This paper introduces the Language Model Linguistic Personality Assessment (LMLPA), a system designed to evaluate the linguistic personalities of LLMs. Our system helps to understand LLMs' language generation capabilities by quantitatively assessing the distinct personality traits reflected in their linguistic outputs. Unlike traditional human-centric psychometrics, the LMLPA adapts a personality assessment questionnaire, specifically the Big Five Inventory, to align with the operational capabilities of LLMs, and also incorporates the findings from previous language-based personality measurement literature. To mitigate sensitivity to the order of options, our questionnaire is designed to be open-ended, resulting in textual answers. Thus, the AI rater is needed to transform ambiguous personality information from text responses into clear numerical indicators of personality traits. Utilising Principal Component Analysis and reliability validations, our findings demonstrate that LLMs possess distinct personality traits that can be effectively quantified by the LMLPA. This research contributes to Human-Computer Interaction and Human-Centered AI, providing a robust framework for future studies to refine AI personality assessments and expand their applications in multiple areas, including education and manufacturing.

翻译：大型语言模型（LLM）在日常生活和研究中的应用日益广泛。其中最常见的用例之一是利用LLM的语言生成能力实现对话交互。正如两个人类之间的对话一样，由LLM驱动的实体与人类之间的对话也取决于对话者的个性特征。然而，目前衡量特定LLM的个性仍面临挑战。本文介绍了语言模型语言人格评估系统（LMLPA），该系统旨在评估LLM的语言个性。通过定量评估LLM语言输出中反映的独特人格特质，我们的系统有助于理解LLM的语言生成能力。与传统的以人为中心的心理测量方法不同，LMLPA调整了人格评估问卷（特别是大五人格量表），使其与LLM的操作能力相匹配，并整合了先前基于语言的人格测量文献的研究成果。为降低对选项顺序的敏感性，我们的问卷设计为开放式问题，从而获得文本形式的答案。因此，需要借助AI评估器将文本回答中模糊的人格信息转化为清晰的人格特质数值指标。通过主成分分析和信效度验证，我们的研究结果表明，LLM具有可通过LMLPA有效量化的独特人格特质。这项研究为人机交互和以人为中心的人工智能领域作出贡献，为未来研究提供了完善AI人格评估的稳健框架，并拓展了其在教育和制造等多个领域的应用前景。