Managing heterogeneous datasets that vary in complexity, size, and similarity in continual learning presents a significant challenge. Task-agnostic continual learning is necessary to address this challenge, as datasets with varying similarity pose difficulties in distinguishing task boundaries. Conventional task-agnostic continual learning practices typically rely on rehearsal or regularization techniques. However, rehearsal methods may struggle with varying dataset sizes and regulating the importance of old and new data due to rigid buffer sizes. Meanwhile, regularization methods apply generic constraints to promote generalization but can hinder performance when dealing with dissimilar datasets lacking shared features, necessitating a more adaptive approach. In this paper, we propose AdaptCL, a novel adaptive continual learning method to tackle heterogeneity in sequential datasets. AdaptCL employs fine-grained data-driven pruning to adapt to variations in data complexity and dataset size. It also utilizes task-agnostic parameter isolation to mitigate the impact of varying degrees of catastrophic forgetting caused by differences in data similarity. Through a two-pronged case study approach, we evaluate AdaptCL on both datasets of MNIST Variants and DomainNet, as well as datasets from different domains. The latter include both large-scale, diverse binary-class datasets and few-shot, multi-class datasets. Across all these scenarios, AdaptCL consistently exhibits robust performance, demonstrating its flexibility and general applicability in handling heterogeneous datasets.

翻译：在持续学习中管理复杂程度、规模及相似度各异的异质性数据集是一项重大挑战。由于相似度不同的数据集难以区分任务边界，任务不可知的持续学习成为应对该挑战的必要手段。传统任务不可知持续学习通常依赖重放或正则化技术。然而，重放方法可能因固定缓冲池大小而难以适应数据集规模变化，且无法有效调节新旧数据的重要性。同时，正则化方法通过施加通用约束促进泛化能力，但在处理缺乏共享特征的非相似数据集时可能影响性能，因此需要更具适应性的方法。本文提出AdaptCL——一种新颖的自适应持续学习方法，旨在解决序列数据集的异质性。AdaptCL采用细粒度的数据驱动剪枝策略，自适应数据复杂度与数据集规模的变化；并利用任务不可知的参数隔离机制，缓解因数据相似度差异导致的不同程度灾难性遗忘。通过双管齐下的案例研究方法，我们在MNIST变体数据集和DomainNet数据集，以及跨领域数据集上评估了AdaptCL。后者既包含大规模多样化的二分类数据集，也涵盖小样本多分类数据集。在所有场景中，AdaptCL均展现出稳定的鲁棒性能，充分证明了其在处理异质性数据集方面的灵活性与通用适用性。