Loss of plasticity is one of the main challenges in continual learning with deep neural networks, where neural networks trained via backpropagation gradually lose their ability to adapt to new tasks and perform significantly worse than their freshly initialized counterparts. The main contribution of this paper is to propose a new hypothesis that experience replay addresses the loss of plasticity in continual learning. Here, experience replay is a form of memory. We provide supporting evidence for this hypothesis. In particular, we demonstrate in multiple different tasks, including regression, classification, and policy evaluation, that by simply adding an experience replay and processing the data in the experience replay with Transformers, the loss of plasticity disappears. Notably, we do not alter any standard components of deep learning. For example, we do not change backpropagation. We do not modify the activation functions. And we do not use any regularization. We conjecture that experience replay and Transformers can address the loss of plasticity because of the in-context learning phenomenon.

翻译：可塑性丧失是深度神经网络持续学习中的主要挑战之一，其中通过反向传播训练的神经网络逐渐丧失适应新任务的能力，其表现显著逊于新初始化的网络。本文的主要贡献在于提出一个新假设：经验回放能够解决持续学习中的可塑性丧失问题。此处，经验回放是一种记忆形式。我们为该假设提供了支持性证据。具体而言，我们在回归、分类和策略评估等多种任务中证明，仅需添加经验回放模块并使用Transformer处理其中的数据，可塑性丧失现象即会消失。值得注意的是，我们未对深度学习的任何标准组件进行修改。例如，我们未改变反向传播算法，未调整激活函数，也未使用任何正则化方法。我们推测经验回放与Transformer能够解决可塑性丧失问题，源于其上下文学习现象。