Large Language Models (LLMs), such as the Generative Pretrained Transformer (GPT), have achieved tremendous success in various language tasks, but their emergent abilities have also raised many questions, concerns, and challenges that need to be addressed. To gain a better understanding of the models' inner mechanisms, we analyze the hidden state and channel wave dynamics in a small GPT, focusing on the coherence of wave patterns in terms of cross-channel correlation and individual auto-correlation. Our findings suggest that wave dynamics offer consistent and repeatable intrinsic oscillation modes, along with context-aware plasticity and expressiveness in language generation. By analyzing wave patterns, coherence, and clustering, we provide a systematic way to identify and interpret the functionality of the hidden state channels, paving the way to understand and control higher-level language pattern formation. In addition, we investigate the Poisson statistics of spelling errors in text sequence generation across various levels of model training and observe a phase-transition-like process. As coherence builds up, there is a competition between the generation of correct and misspelled words. However, once the model is adequately trained and significant coherence has emerged, the coherent process becomes strong enough to effectively suppress spelling errors, preventing the cascade amplification of defects. The distribution of correct spellings transitions from Poissonian to Sub-Poissonian, while the distribution of misspellings shows the opposite trend. By leveraging concepts and techniques from quantum physics, we gain novel insights into the dynamics of the small GPT. This approach can be extended to larger language models that exhibit more complex coherent language patterns, opening up opportunities to interpret their emergent capabilities and develop more specialized models.

翻译：大型语言模型（LLMs），如生成式预训练Transformer（GPT），已在各类语言任务中取得巨大成功，但其涌现能力也引发了诸多需要应对的问题、担忧与挑战。为深入理解模型内部机制，我们分析小型GPT中的隐藏状态与通道波动力学，重点关注波模式在跨通道相关性及个体自相关性方面的相干性。研究结果表明，波动力学提供了可重复且一致的固有振荡模式，并在语言生成中展现出情境感知的可塑性与表现力。通过分析波模式、相干性及聚类，我们提供了一种系统化方法以识别和解释隐藏状态通道的功能，为理解和调控更高层次的语言模式形成奠定基础。此外，我们研究了不同模型训练阶段文本序列生成中拼写错误的泊松统计特性，并观测到类似相变的过程。随着相干性的增强，正确单词与拼写错误单词的生成之间存在竞争。然而，一旦模型得到充分训练且形成显著相干性，相干过程将强大到足以有效抑制拼写错误，阻止缺陷的级联放大。正确拼写的分布从泊松分布转变为亚泊松分布，而拼写错误的分布则呈现相反趋势。通过借鉴量子物理的概念与技术，我们获得了对小型GPT动力学的新见解。该方法可推广至展现更复杂相干语言模式的大型语言模型，为解读其涌现能力及开发更专业化模型开辟了机遇。