Due to the model aging problem, Deep Neural Networks (DNNs) need updates to adjust them to new data distributions. The common practice leverages incremental learning (IL), e.g., Class-based Incremental Learning (CIL) that updates output labels, to update the model with new data and a limited number of old data. This avoids heavyweight training (from scratch) using conventional methods and saves storage space by reducing the number of old data to store. But it also leads to poor performance in fairness. In this paper, we show that CIL suffers both dataset and algorithm bias problems, and existing solutions can only partially solve the problem. We propose a novel framework, CILIATE, that fixes both dataset and algorithm bias in CIL. It features a novel differential analysis guided dataset and training refinement process that identifies unique and important samples overlooked by existing CIL and enforces the model to learn from them. Through this process, CILIATE improves the fairness of CIL by 17.03%, 22.46%, and 31.79% compared to state-of-the-art methods, iCaRL, BiC, and WA, respectively, based on our evaluation on three popular datasets and widely used ResNet models.

翻译：摘要：针对深度神经网络（DNN）因模型老化问题，需更新模型以适应新数据分布。当前普遍采用增量学习（IL），例如通过更新输出标签的基于类别的增量学习（CIL），利用新数据和有限旧数据更新模型。该方法避免了传统方法重新训练的沉重负担，并通过减少旧数据存储量节省存储空间，但同时导致公平性表现不佳。本文表明，CIL面临数据集偏差与算法偏差双重问题，现有解决方案仅能部分缓解。我们提出新型框架CILIATE，同时修复CIL中的数据集偏差与算法偏差。该框架通过创新的差异分析引导数据集与训练优化流程，识别被现有CIL忽略的独特关键样本，并强制模型从中学习。基于三个主流数据集与广泛使用的ResNet模型的评估表明，CILIATE相较于当前最优方法iCaRL、BiC和WA，分别将CIL的公平性提升17.03%、22.46%和31.79%。