Transformer neural networks can exhibit a surprising capacity for in-context learning (ICL) despite not being explicitly trained for it. Prior work has provided a deeper understanding of how ICL emerges in transformers, e.g. through the lens of mechanistic interpretability, Bayesian inference, or by examining the distributional properties of training data. However, in each of these cases, ICL is treated largely as a persistent phenomenon; namely, once ICL emerges, it is assumed to persist asymptotically. Here, we show that the emergence of ICL during transformer training is, in fact, often transient. We train transformers on synthetic data designed so that both ICL and in-weights learning (IWL) strategies can lead to correct predictions. We find that ICL first emerges, then disappears and gives way to IWL, all while the training loss decreases, indicating an asymptotic preference for IWL. The transient nature of ICL is observed in transformers across a range of model sizes and datasets, raising the question of how much to "overtrain" transformers when seeking compact, cheaper-to-run models. We find that L2 regularization may offer a path to more persistent ICL that removes the need for early stopping based on ICL-style validation tasks. Finally, we present initial evidence that ICL transience may be caused by competition between ICL and IWL circuits.

翻译：Transformer神经网络即使未经明确训练，也能展现出惊人的上下文学习能力。先前研究通过机械可解释性、贝叶斯推断或训练数据分布特性等视角，加深了我们对Transformer中上下文学习涌现机制的理解。然而在这些研究中，上下文学习基本被视作持续性现象——即一旦涌现，便会渐近保持。本文表明，在Transformer训练过程中，上下文学习的涌现实际上常具有暂态特性。我们使用合成数据训练Transformer，这类数据使得上下文学习与权重内学习策略均能产生正确预测。研究发现，上下文学习先涌现，随后消失并被权重内学习取代，过程中训练损失持续下降，表明模型渐近倾向于权重内学习。这一暂态特性在不同模型规模和数据集上的Transformer中均有体现，引发了一个问题：在追求更紧凑、低成本模型时，应在多大程度上对Transformer进行"过度训练"。我们发现L2正则化或能形成更持久的上下文学习，从而无需基于上下文学习型验证任务进行早停。最后，我们初步证明上下文学习的暂态性可能源于其与权重内学习间的电路竞争。