Transformer neural networks can exhibit a surprising capacity for in-context learning (ICL) despite not being explicitly trained for it. Prior work has provided a deeper understanding of how ICL emerges in transformers, e.g. through the lens of mechanistic interpretability, Bayesian inference, or by examining the distributional properties of training data. However, in each of these cases, ICL is treated largely as a persistent phenomenon; namely, once ICL emerges, it is assumed to persist asymptotically. Here, we show that the emergence of ICL during transformer training is, in fact, often transient. We train transformers on synthetic data designed so that both ICL and in-weights learning (IWL) strategies can lead to correct predictions. We find that ICL first emerges, then disappears and gives way to IWL, all while the training loss decreases, indicating an asymptotic preference for IWL. The transient nature of ICL is observed in transformers across a range of model sizes and datasets, raising the question of how much to "overtrain" transformers when seeking compact, cheaper-to-run models. We find that L2 regularization may offer a path to more persistent ICL that removes the need for early stopping based on ICL-style validation tasks. Finally, we present initial evidence that ICL transience may be caused by competition between ICL and IWL circuits.

翻译：Transformer神经网络尽管未经显式训练，却可能展现出令人惊讶的上下文学习（ICL）能力。先前研究通过机械可解释性、贝叶斯推断或训练数据分布属性分析等视角，加深了对Transformer中ICL涌现机制的理解。然而，这些研究大多将ICL视为持续性现象——即一旦ICL涌现，便假定其会渐近持续存在。本文证明，在Transformer训练过程中，ICL的涌现实际上往往具有瞬态特性。我们在合成数据上训练Transformer，这些数据的设计使得ICL与权重内学习（IWL）策略都能实现正确预测。研究发现，ICL首先涌现，随后消失并被IWL取代，而训练损失全程持续下降，表明模型渐近趋向于IWL。这种ICL瞬态特性在多种模型规模与数据集的Transformer中均被观测到，这引发了一个问题：在追求紧凑且运行成本更低模型时，究竟需要多大程度地"过度训练"Transformer。我们发现L2正则化可能为持久化ICL提供途径，从而消除基于ICL风格验证任务进行早停的必要性。最后，我们提出的初步证据表明，ICL的瞬态特性可能源于ICL与IWL回路间的竞争。