Questionnaires are a common method for detecting the personality of Large Language Models (LLMs). However, their reliability is often compromised by two main issues: hallucinations (where LLMs produce inaccurate or irrelevant responses) and the sensitivity of responses to the order of the presented options. To address these issues, we propose combining text mining with questionnaires method. Text mining can extract psychological features from the LLMs' responses without being affected by the order of options. Furthermore, because this method does not rely on specific answers, it reduces the influence of hallucinations. By normalizing the scores from both methods and calculating the root mean square error, our experiment results confirm the effectiveness of this approach. To further investigate the origins of personality traits in LLMs, we conduct experiments on both pre-trained language models (PLMs), such as BERT and GPT, as well as conversational models (ChatLLMs), such as ChatGPT. The results show that LLMs do contain certain personalities, for example, ChatGPT and ChatGLM exhibit the personality traits of 'Conscientiousness'. Additionally, we find that the personalities of LLMs are derived from their pre-trained data. The instruction data used to train ChatLLMs can enhance the generation of data containing personalities and expose their hidden personality. We compare the results with the human average personality score, and we find that the personality of FLAN-T5 in PLMs and ChatGPT in ChatLLMs is more similar to that of a human, with score differences of 0.34 and 0.22, respectively.

翻译：问卷调查是检测大型语言模型（LLMs）个性的常用方法。然而，其可靠性常受两个主要问题影响：幻觉（即LLMs产生不准确或不相关的回答）以及回答对选项呈现顺序的敏感性。为解决这些问题，我们提出将文本挖掘与问卷调查方法相结合。文本挖掘能够从LLMs的回应中提取心理特征，且不受选项顺序影响。此外，由于该方法不依赖特定答案，因此减少了幻觉的干扰。通过对两种方法的得分进行归一化处理并计算均方根误差，我们的实验结果证实了该方法的有效性。为进一步探究LLMs个性特征的来源，我们对预训练语言模型（PLMs，如BERT和GPT）和对话模型（ChatLLMs，如ChatGPT）进行了实验。结果表明，LLMs确实包含某些个性特征，例如ChatGPT和ChatGLM表现出“尽责性”的人格特质。此外，我们发现LLMs的个性源于其预训练数据。用于训练ChatLLMs的指令数据能够增强包含个性的数据生成，并暴露其隐藏的个性特征。我们将结果与人类平均个性得分进行比较，发现PLMs中的FLAN-T5和ChatLLMs中的ChatGPT的个性更接近人类，得分差异分别为0.34和0.22。