This paper focuses on an under-explored yet important problem: Federated Class-Continual Learning (FCCL), where new classes are dynamically added in federated learning. Existing FCCL works suffer from various limitations, such as requiring additional datasets or storing the private data from previous tasks. In response, we first demonstrate that non-IID data exacerbates catastrophic forgetting issue in FL. Then we propose a novel method called TARGET (federat\textbf{T}ed cl\textbf{A}ss-continual lea\textbf{R}nin\textbf{G} via \textbf{E}xemplar-free dis\textbf{T}illation), which alleviates catastrophic forgetting in FCCL while preserving client data privacy. Our proposed method leverages the previously trained global model to transfer knowledge of old tasks to the current task at the model level. Moreover, a generator is trained to produce synthetic data to simulate the global distribution of data on each client at the data level. Compared to previous FCCL methods, TARGET does not require any additional datasets or storing real data from previous tasks, which makes it ideal for data-sensitive scenarios.
翻译:本文聚焦于一个尚未充分探索但至关重要的问题:联邦类持续学习(FCCL),即在联邦学习中动态添加新类别。现有FCCL方法存在诸多局限,例如需要额外数据集或存储先前任务中的私有数据。为此,我们首先证明非独立同分布数据会加剧联邦学习中的灾难性遗忘问题,继而提出一种名为TARGET(通过无样本蒸馏的联邦类持续学习)的新方法,可在保护客户端数据隐私的同时缓解FCCL中的灾难性遗忘。所提方法利用预训练的全局模型,在模型层面将旧任务知识迁移至当前任务;同时在数据层面训练生成器产生合成数据,模拟每个客户端上的全局数据分布。与以往FCCL方法相比,TARGET无需额外数据集或存储先前任务的真实数据,因此特别适用于数据敏感场景。