Federated Learning (FL) enables collaborative model training among participants while guaranteeing the privacy of raw data. Mainstream FL methodologies overlook the dynamic nature of real-world data, particularly its tendency to grow in volume and diversify in classes over time. This oversight results in FL methods suffering from catastrophic forgetting, where the trained models inadvertently discard previously learned information upon assimilating new data. In response to this challenge, we propose a novel Federated Class-Incremental Learning (FCIL) method, named \underline{Fed}erated \underline{C}lass-Incremental \underline{L}earning with New-Class \underline{A}ugmented \underline{S}elf-Di\underline{S}tillation (FedCLASS). The core of FedCLASS is to enrich the class scores of historical models with new class scores predicted by current models and utilize the combined knowledge for self-distillation, enabling a more sufficient and precise knowledge transfer from historical models to current models. Theoretical analyses demonstrate that FedCLASS stands on reliable foundations, considering scores of old classes predicted by historical models as conditional probabilities in the absence of new classes, and the scores of new classes predicted by current models as the conditional probabilities of class scores derived from historical models. Empirical experiments demonstrate the superiority of FedCLASS over four baseline algorithms in reducing average forgetting rate and boosting global accuracy.
翻译:联邦学习使得参与者能够协同训练模型,同时保证原始数据的隐私性。主流联邦学习方法忽略了真实数据的动态特性,特别是其随时间增长的规模与类别多样性。这种疏忽导致联邦学习方法遭受灾难性遗忘,即训练模型在吸收新数据时会无意中丢弃先前学到的知识。针对这一挑战,我们提出了一种新型联邦类增量学习方法——基于新类增强自蒸馏的联邦类增量学习(FedCLASS)。FedCLASS的核心思想是用当前模型预测的新类分数丰富历史模型的类别分数,并利用组合知识进行自蒸馏,从而实现从历史模型到当前模型更充分、更精确的知识迁移。理论分析表明,FedCLASS建立在可靠基础上:将历史模型预测的旧类分数视为无新类条件下的条件概率,而当前模型预测的新类分数视为历史模型衍生类别分数的条件概率。实验证明,在降低平均遗忘率和提升全局准确率方面,FedCLASS优于四种基线算法。