Grokking, or delayed generalization after memorization, incurs substantial training steps across algebraic tasks. Cross-operator transfer from a grokked donor accelerates target acquisition, yet post-grokking AdamW continuation triggers test-accuracy relapse as unannealed updates drift across residual interfaces and decaying second moments amplify late gradients. Across depths one to three on modular arithmetic ($\mathbb{Z}_{113}$), non-Abelian $S_5$ conjugation, and real-sensory MNIST modular arithmetic, we separate representational transfer from preconditioner-governed consolidation across thirteen experimental batteries. First, co-transferring internal blocks accelerates one-layer and two-layer modular takeoff beyond boundary interfaces, while graded prefix pinning bounds depth-three drift under unannealed continuation. Second, an iso-parameter $S_5$ scaling grid separates attention head dimension from MLP width: widening two-head attention to dimension $64$ reaches $98.54\%$ cold accuracy at $427\mathrm{k}$ parameters and $85.95\%$ boundary-only transfer, while final-slot representations refine conjugation directly across blocks without an intermediate binary-product bottleneck. Third, Plateau-Gated Leaky Consolidation pairs validation-envelope step damping with leaky second-moment decay, reaching $84.85\%$ zero-leakage validation-gated risk-constrained accuracy across five pre-saturation two-layer $S_5$ suites while cutting step budget relative to hard anchoring. Fourth, transferring a grokked visual trunk and blocks to image-disjoint MNIST targets reaches $92.27\%$ and $90.76\%$ accuracy, prunes odd centroid harmonics, and reaches $90.42\%$ risk-constrained accuracy across $60/60$ branches without hyperparameter retuning.
翻译:暂无翻译