Prior studies on the emergence in large models have primarily focused on how the functional capabilities of large language models (LLMs) scale with model size. Our research, however, transcends this traditional paradigm, aiming to deepen our understanding of the emergence within LLMs by placing a special emphasis not just on the model size but more significantly on the complex behavior of neuron interactions during the training process. By introducing the concepts of "self-organization" and "multifractal analysis," we explore how neuron interactions dynamically evolve during training, leading to "emergence," mirroring the phenomenon in natural systems where simple micro-level interactions give rise to complex macro-level behaviors. To quantitatively analyze the continuously evolving interactions among neurons in large models during training, we propose the Neuron-based Multifractal Analysis (NeuroMFA). Utilizing NeuroMFA, we conduct a comprehensive examination of the emergent behavior in LLMs through the lens of both model size and training process, paving new avenues for research into the emergence in large models.

翻译：先前关于大模型涌现现象的研究主要聚焦于大语言模型功能能力随模型规模的扩展规律。然而，本研究超越这一传统范式，旨在通过不仅关注模型规模，更着重分析训练过程中神经元交互的复杂行为，深化对大型语言模型涌现机制的理解。通过引入"自组织"与"多重分形分析"概念，我们探索了训练过程中神经元交互的动态演化如何引发"涌现"现象，这与自然系统中微观层面的简单交互产生宏观层面复杂行为的过程具有相似性。为定量分析大模型训练过程中神经元间持续演化的交互模式，我们提出基于神经元的多元分形分析方法。借助该方法，我们从模型规模与训练进程双重视角对大语言模型中的涌现行为展开全面剖析，为大型模型涌现现象研究开辟了新路径。