Emergent intelligence have played a major role in the modern AI development. While existing studies primarily rely on empirical observations to characterize this phenomenon, a rigorous theoretical framework remains underexplored. This study attempts to develop a mathematical approach to formalize emergent intelligence from the perspective of limit theory. Specifically, we introduce a performance function E(N, P, K), dependent on data size N, model size P and training steps K, to quantify intelligence behavior. We posit that intelligence emerges as a transition from finite to effectively infinite knowledge, and thus recast emergent intelligence as existence of the limit $\lim_{N,P,K \to \infty} \mathcal{E}(N,P,K)$, with emergent abilities corresponding to the limiting behavior. This limit theory helps reveal that emergent intelligence originates from the existence of a parameter-limit architecture (referred to as the limit architecture), and that emergent intelligence rationally corresponds to the learning behavior of this limit system. By introducing tools from nonlinear Lipschitz operator theory, we prove that the necessary and sufficient conditions for existence of the limit architecture. Furthermore, we derive the scaling law of foundation models by leveraging tools of Lipschitz operator and covering number. Theoretical results show that: 1) emergent intelligence is governed by three key factors-training steps, data size and the model architecture, where the properties of basic blocks play a crucial role in constructing foundation models; 2) the critical condition Lip(T)=1 for emergent intelligence provides theoretical support for existing findings. 3) emergent intelligence is determined by an infinite-dimensional system, yet can be effectively realized in practice through a finite-dimensional architecture. Our empirical results corroborate these theoretical findings.

翻译：涌现智能在现代人工智能发展中发挥了重要作用。现有研究主要通过经验观察来描述这一现象，但严格的理论框架仍待探索。本研究尝试从极限理论视角建立数学方法来形式化涌现智能。具体而言，我们引入依赖于数据规模N、模型规模P和训练步数K的性能函数E(N,P,K)来量化智能行为。我们提出智能表现为从有限知识向无限知识的知识转化，因此将涌现智能重构为极限$\lim_{N,P,K \to \infty} \mathcal{E}(N,P,K)$的存在性，其涌现能力对应极限行为。该极限理论揭示：涌现智能源于参数-极限架构（简称极限架构）的存在性，且涌现智能理性地对应这个极限系统的学习行为。通过引入非线性Lipschitz算子理论工具，我们证明了极限架构存在的充要条件。进一步地，我们利用Lipschitz算子和覆盖数工具推导了基础模型的缩放定律。理论结果表明：1）涌现智能受三个关键因素控制——训练步数、数据规模和模型架构，其中基础模块的性质在构建基础模型中起关键作用；2）涌现智能的临界条件Lip(T)=1为现有发现提供了理论支撑；3）涌现智能由无限维系统决定，但可通过有限维架构在实践中有效实现。我们的实证结果佐证了这些理论发现。