This work investigates the entanglement between Continual Learning (CL) and Transfer Learning (TL). In particular, we shed light on the widespread application of network pretraining, highlighting that it is itself subject to catastrophic forgetting. Unfortunately, this issue leads to the under-exploitation of knowledge transfer during later tasks. On this ground, we propose Transfer without Forgetting (TwF), a hybrid approach building upon a fixed pretrained sibling network, which continuously propagates the knowledge inherent in the source domain through a layer-wise loss term. Our experiments indicate that TwF steadily outperforms other CL methods across a variety of settings, averaging a 4.81% gain in Class-Incremental accuracy over a variety of datasets and different buffer sizes.

翻译：这项工作调查了持续学习(CL)和转移学习(TL)之间的纠缠关系。 特别是,我们阐明了网络预培训(TL)的广泛应用,强调网络预培训本身会受到灾难性的遗忘。 不幸的是,这个问题导致在以后的任务中知识转让的利用不足。 在此基础上,我们提议不忘转让(TwF),这是建立在固定的预先培训的编织网基础上的混合方法,通过一个分层损失术语,不断传播源域固有的知识。 我们的实验表明,TwF在不同环境中稳步地超越了其他CL方法,平均在各种数据集和不同缓冲大小的等级偏差精度方面增加了4.81%。