Capability emergence during neural network training remains mechanistically opaque. We track five geometric measures across five model scales (405K-85M parameters), 120+ emergence events in eight algorithmic tasks, and three Pythia language models (160M-2.8B). We find: (1) training begins with a universal representation collapse to task-specific floors that are scale-invariant across a 210X parameter range (e.g., modular arithmetic collapses to RANKME ~ 2.0 regardless of model size); (2) collapse propagates top-down through layers (32/32 task X model consistency), contradicting bottom-up feature-building intuition; (3) a geometric hierarchy in which representation geometry leads emergence (75-100% precursor rate for hard tasks), while the local learning coefficient is synchronous (0/24 precursor) and Hessian measures lag. We also delineate prediction limits: geometric measures encode coarse task difficulty but not fine-grained timing (within-class concordance 27%; when task ordering reverses across scales, prediction fails at 26%). On Pythia, global geometric patterns replicate but per-task precursor signals do not -- the precursor relationship requires task-training alignment that naturalistic pre-training does not provide. Our contribution is the geometric anatomy of emergence and its boundary conditions, not a prediction tool.

翻译：神经网络训练过程中的能力涌现机制仍不明确。本研究追踪了五个几何度量在五个模型规模（405K-8500万参数）、八个算法任务中的120余次涌现事件以及三个Pythia语言模型（1.6亿-28亿参数）中的表现。研究发现：（1）训练起始阶段普遍存在表示坍缩现象，坍缩至任务特定的稳定水平，且该水平在210倍参数范围内呈现尺度不变性（例如模运算任务中RANKME值始终坍缩至约2.0，与模型规模无关）；（2）坍缩过程以自上而下的方式在神经网络层间传播（在32/32的任务×模型组合中表现一致），这与自下而上的特征构建直觉相悖；（3）存在几何层级关系：表示几何结构引领能力涌现（在困难任务中的先导率达75-100%），而局部学习系数呈同步变化（0/24先导率），Hessian度量则存在滞后。同时界定了预测边界：几何度量可编码任务的宏观难度但无法预测精细时间节点（类内一致性仅27%；当任务排序在不同规模间反转时，预测失败率达26%）。在Pythia模型中，全局几何模式可复现但具体任务的先导信号消失——先导关系需要任务与训练的精确对齐，而自然主义的预训练无法提供这种对齐条件。本研究的贡献在于揭示了能力涌现的几何解剖结构及其边界条件，而非提供预测工具。