Research into the external behaviors and internal mechanisms of large language models (LLMs) has shown promise in addressing complex tasks in the physical world. Studies suggest that powerful LLMs, like GPT-4, are beginning to exhibit human-like cognitive abilities, including planning, reasoning, and reflection. In this paper, we introduce a research line and methodology called LLM Psychology, leveraging human psychology experiments to investigate the cognitive behaviors and mechanisms of LLMs. We migrate the Typoglycemia phenomenon from psychology to explore the "mind" of LLMs. Unlike human brains, which rely on context and word patterns to comprehend scrambled text, LLMs use distinct encoding and decoding processes. Through Typoglycemia experiments at the character, word, and sentence levels, we observe: (I) LLMs demonstrate human-like behaviors on a macro scale, such as lower task accuracy and higher token/time consumption; (II) LLMs exhibit varying robustness to scrambled input, making Typoglycemia a benchmark for model evaluation without new datasets; (III) Different task types have varying impacts, with complex logical tasks (e.g., math) being more challenging in scrambled form; (IV) Each LLM has a unique and consistent "cognitive pattern" across tasks, revealing general mechanisms in its psychology process. We provide an in-depth analysis of hidden layers to explain these phenomena, paving the way for future research in LLM Psychology and deeper interpretability.

翻译：对大型语言模型外部行为与内部机制的研究，已展现出解决物理世界复杂任务的潜力。研究表明，以GPT-4为代表的强大LLM正开始呈现类人的认知能力，包括规划、推理与反思。本文提出名为"LLM心理学"的研究路径与方法论，借鉴人类心理学实验来探究LLM的认知行为与机制。我们将心理学中的Typoglycemia现象迁移至LLM研究领域，以此探索其"思维"本质。与人类大脑依赖上下文和词汇模式理解乱序文本不同，LLM采用独特的编码-解码处理机制。通过在字符、词汇和句子三个层面进行的Typoglycemia实验，我们观察到：（I）在宏观层面，LLM表现出类人行为特征，如任务准确率下降、token/时间消耗增加；（II）不同模型对乱序输入具有差异化的鲁棒性，使Typoglycemia成为无需新数据集的模型评估基准；（III）任务类型的影响存在差异，复杂逻辑任务（如数学计算）在乱序形式下更具挑战性；（IV）每个LLM在跨任务中均展现出独特且稳定的"认知模式"，揭示了其心理处理过程中的通用机制。我们通过对隐藏层的深入分析解释这些现象，为LLM心理学研究的深化及模型可解释性探索开辟了新路径。