Transformers trained on modular arithmetic exhibit sharp transitions between memorization, generalization, and collapse. We show that weight decay acts as a scalar empirical control parameter for these regimes, and introduce two cheap online diagnostics, mean pairwise attention-head cosine similarity and entropy standard deviation, that track training dynamics from attention activations alone and complement loss-landscape diagnostics at lower compute cost. Across eleven experimental conditions and three model scales (0.82M to 85M parameters), the weight-decay axis separates memorization, developmental grokking, and collapse. A near-transition logistic fit localizes the memorization-to-developmental boundary at $λ_c=0.0158$ (95% CI [0.0109, 0.0200], N=210); a power-law fit gives an empirical exponent $ν=0.757$ (CI [0.725, 0.799]). Reference exponents $ν=1/2$ and 3D Ising $ν\approx 0.63$ lie outside this empirical CI under our four-bin grid, so we report $ν$ as empirical and defer universality-class identification to denser finite-size-scaling work. A horizon-matched multi-task replication (n=280, four modular operations) preserves the weight-decay control pattern; a paired attention-head re-initialization experiment at $λ=0.05$ changes Phase-2 amplitude (Cohen's $d=-1.190$, n=10, $p_t=4.5 \times 10^{-3}$), while matched weight-norm clipping does not. Three cross-architecture probes (4L MLP, 4L LSTM, and 4L Mamba; each n=70) replicate the weight-decay-controlled transition with architecture-specific $λ_c$ values. Main diagnostic claims are scoped to modular arithmetic in small transformer attention models; the non-attention experiments are scope probes, and architecture-wide, language-model, and universality-class claims are out of scope.

翻译：在模算术任务上训练的Transformer在记忆化、泛化与坍塌三种状态间展现出尖锐的相变行为。我们发现权重衰减可作为这些状态标量经验控制参数，并提出两种低成本在线诊断指标——平均逐注意力头余弦相似度与熵标准差——仅通过注意力激活值即可追踪训练动态，并以更低计算代价补足损失景观诊断方法的不足。在涵盖11种实验条件、三种模型规模（0.82M至85M参数）的测试中，权重衰减轴清晰区分了记忆化、发展性顿悟与坍塌三种状态。近相变逻辑斯蒂拟合将记忆化-发展性边界定位在λ_c=0.0158（95%置信区间[0.0109, 0.0200]，N=210）；幂律拟合给出经验指数ν=0.757（置信区间[0.725, 0.799]）。在四窗口网格下，参考指数ν=1/2及三维伊辛模型ν≈0.63均落于此经验置信区间外，故我们将ν报告为经验值，并将普适性类别判定工作留待更密集的有限尺寸标度研究。跨水平多任务复制实验（n=280，四种模运算）验证了权重衰减控制模式；在λ=0.05条件下进行的配对注意力头重初始化实验改变了阶段2幅度（Cohen's d=-1.190，n=10，p_t=4.5×10^{-3}），而匹配的权重范数裁剪实验则未呈现此效应。三种跨架构探针实验（4层MLP、4层LSTM与4层Mamba，各n=70）均复制了权重衰减控制的相变行为，并得到架构特定的λ_c值。主要诊断结论限定于小型Transformer注意力模型中的模算术任务；非注意力实验属于架构探针范畴，而关于全架构、语言模型及普适性类别的判定超出本文研究范围。