Most continual learning (CL) algorithms have focused on tackling the stability-plasticity dilemma, that is, the challenge of preventing the forgetting of previous tasks while learning new ones. However, they have overlooked the impact of the knowledge transfer when the dataset in a certain task is biased - namely, when some unintended spurious correlations of the tasks are learned from the biased dataset. In that case, how would they affect learning future tasks or the knowledge already learned from the past tasks? In this work, we carefully design systematic experiments using one synthetic and two real-world datasets to answer the question from our empirical findings. Specifically, we first show through two-task CL experiments that standard CL methods, which are unaware of dataset bias, can transfer biases from one task to another, both forward and backward, and this transfer is exacerbated depending on whether the CL methods focus on the stability or the plasticity. We then present that the bias transfer also exists and even accumulate in longer sequences of tasks. Finally, we propose a simple, yet strong plug-in method for debiasing-aware continual learning, dubbed as Group-class Balanced Greedy Sampling (BGS). As a result, we show that our BGS can always reduce the bias of a CL model, with a slight loss of CL performance at most.

翻译：大多数持续学习算法主要关注解决稳定性-可塑性困境，即防止遗忘先前任务的同时学习新任务的挑战。然而，当某个任务的数据集存在偏差（即从有偏数据集中学习到任务间非预期的虚假相关性）时，这些算法忽视了知识迁移的影响。在这种情况下，偏差会如何影响未来任务的学习或已习得的先前任务知识？本文中，我们精心设计系统性实验，采用一个合成数据集和两个真实世界数据集，通过实证研究回答这一问题。具体而言，首先通过双任务持续学习实验表明：标准持续学习方法（未感知数据集偏差）会以前向和后向方式将偏差从一个任务迁移至另一个任务，且这种迁移的加剧程度取决于持续学习方法是侧重于稳定性还是可塑性。接着证明，偏差迁移在更长任务序列中仍然存在甚至持续累积。最后，我们提出一种简单而强大的即插即用方法——群类平衡贪婪采样（BGS），专用于消除偏差的持续学习。实验表明，我们的BGS方法总能降低持续学习模型的偏差，至多仅以轻微的持续学习性能损失为代价。