Class-Incremental Learning aims to update a deep classifier to learn new categories while maintaining or improving its accuracy on previously observed classes. Common methods to prevent forgetting previously learned classes include regularizing the neural network updates and storing exemplars in memory, which come with hyperparameters such as the learning rate, regularization strength, or the number of exemplars. However, these hyperparameters are usually only tuned at the start and then kept fixed throughout the learning sessions, ignoring the fact that newly encountered tasks may have varying levels of novelty or difficulty. This study investigates the necessity of hyperparameter `adaptivity' in Class-Incremental Learning: the ability to dynamically adjust hyperparameters such as the learning rate, regularization strength, and memory size according to the properties of the new task at hand. We propose AdaCL, a Bayesian Optimization-based approach to automatically and efficiently determine the optimal values for those parameters with each learning task. We show that adapting hyperpararmeters on each new task leads to improvement in accuracy, forgetting and memory. Code is available at https://github.com/ElifCerenGokYildirim/AdaCL.

翻译：类增量学习旨在更新深度分类器以学习新类别，同时保持或提高其在先前观察到的类别上的准确性。防止遗忘先前学习类别的常用方法包括正则化神经网络更新以及在内存中存储样本，这些方法涉及学习率、正则化强度或样本数量等超参数。然而，这些超参数通常仅在开始时进行调整，随后在整个学习过程中保持固定，忽略了新遇到的任务可能具有不同程度的新颖性或难度这一事实。本研究探讨了类增量学习中超参数“自适应性”的必要性：即根据当前新任务的性质动态调整学习率、正则化强度和内存大小等超参数的能力。我们提出了AdaCL，一种基于贝叶斯优化的方法，能够自动高效地为每个学习任务确定这些参数的最优值。我们证明，在每个新任务上自适应调整超参数能够提升准确性、减少遗忘并优化内存使用。代码可在 https://github.com/ElifCerenGokYildirim/AdaCL 获取。