Capability emergence during neural network training remains mechanistically opaque. We track five geometric measures across five model scales (405K-85M parameters), 120+ emergence events in eight algorithmic tasks, and three Pythia language models (160M-2.8B). We find: (1) training begins with a universal representation collapse to task-specific floors that are scale-invariant across a 210X parameter range (e.g., modular arithmetic collapses to RANKME $\approx$ 2.0 regardless of model size); (2) collapse propagates top-down through layers (32/32 task X model consistency), contradicting bottom-up feature-building intuition; (3) a geometric hierarchy in which representation geometry leads emergence (75-100% precursor rate for hard tasks), while the local learning coefficient is synchronous (0/24 precursor) and Hessian measures lag. We also delineate prediction limits: geometric measures encode coarse task difficulty but not fine-grained timing (within-class concordance 27%; when task ordering reverses across scales, prediction fails at 26%). On Pythia, global geometric patterns replicate but per-task precursor signals do not -- the precursor relationship requires task-training alignment that naturalistic pre-training does not provide. Our contribution is the geometric anatomy of emergence and its boundary conditions, not a prediction tool.

翻译：神经网络训练过程中的能力涌现机制仍不明确。本研究追踪了五种几何度量在五个模型规模（405K-85M参数）、八个算法任务中的120余次涌现事件以及三个Pythia语言模型（160M-2.8B）中的表现。研究发现：（1）训练起始阶段普遍存在表征坍缩现象，坍缩至任务特定的稳定水平，且该水平在210倍参数范围内呈现尺度不变性（例如模运算任务中RANKME值始终坍缩至≈2.0，与模型规模无关）；（2）坍缩过程沿网络层自上而下传播（在32/32的任务×模型组合中表现一致），这与自下而上的特征构建直觉相悖；（3）存在几何层级关系：表征几何结构主导涌现过程（困难任务的前兆率高达75-100%），而局部学习系数呈同步变化（前兆率为0/24），Hessian相关度量则存在滞后。同时界定了预测边界：几何度量能编码任务的宏观难度，但无法预测细粒度时间特征（类内一致性仅27%；当任务顺序在不同规模间反转时，预测失败率达26%）。在Pythia模型中，全局几何模式可复现，但任务级前兆信号消失——前兆关系需要任务与训练的精确对齐，而自然主义的预训练无法提供这种对齐。本研究的贡献在于揭示涌现现象的几何解剖结构及其边界条件，而非提供预测工具。