Prior studies on the emergence in large models have primarily focused on how the functional capabilities of large language models (LLMs) scale with model size. Our research, however, transcends this traditional paradigm, aiming to deepen our understanding of the emergence within LLMs by placing a special emphasis not just on the model size but more significantly on the complex behavior of neuron interactions during the training process. By introducing the concepts of "self-organization" and "multifractal analysis," we explore how neuron interactions dynamically evolve during training, leading to "emergence," mirroring the phenomenon in natural systems where simple micro-level interactions give rise to complex macro-level behaviors. To quantitatively analyze the continuously evolving interactions among neurons in large models during training, we propose the Neuron-based Multifractal Analysis (NeuroMFA). Utilizing NeuroMFA, we conduct a comprehensive examination of the emergent behavior in LLMs through the lens of both model size and training process, paving new avenues for research into the emergence in large models.

翻译：先前关于大模型涌现的研究主要关注大型语言模型（LLMs）的功能能力如何随模型规模扩展。然而，我们的研究超越了这一传统范式，旨在通过不仅强调模型规模，更突出训练过程中神经元交互的复杂行为，来深化对LLMs中涌现现象的理解。通过引入“自组织”与“多重分形分析”概念，我们探讨了训练过程中神经元交互如何动态演化，从而引发“涌现”，这镜像了自然系统中简单微观交互产生复杂宏观行为的过程。为定量分析大模型训练过程中神经元间持续演化的交互，我们提出了基于神经元的多重分形分析（NeuroMFA）。利用NeuroMFA，我们从模型规模与训练过程双重视角对LLMs的涌现行为进行了全面考察，为大模型涌现研究开辟了新途径。