In Federated Learning (FL), the data in each client is typically assumed fixed or static. However, data often comes in an incremental manner in real-world applications, where the data domain may increase dynamically. In this work, we study catastrophic forgetting with data heterogeneity in Federated Incremental Learning (FIL) scenarios where edge clients may lack enough storage space to retain full data. We propose to employ a simple, generic framework for FIL named Re-Fed, which can coordinate each client to cache important samples for replay. More specifically, when a new task arrives, each client first caches selected previous samples based on their global and local importance. Then, the client trains the local model with both the cached samples and the samples from the new task. Theoretically, we analyze the ability of Re-Fed to discover important samples for replay thus alleviating the catastrophic forgetting problem. Moreover, we empirically show that Re-Fed achieves competitive performance compared to state-of-the-art methods.

翻译：在联邦学习（FL）中，通常假设每个客户端的数据是固定或静态的。然而，在实际应用中，数据往往以增量方式出现，数据域可能动态增长。本研究探讨了联邦增量学习（FIL）场景下数据异构性导致的灾难性遗忘问题，该场景中边缘客户端可能缺乏足够的存储空间保留完整数据。我们提出了一种名为Re-Fed的简单通用FIL框架，该框架能够协调各客户端缓存重要样本进行回放。具体而言，当新任务到达时，每个客户端首先根据样本的全局重要性和局部重要性缓存历史样本，随后利用缓存样本与新任务样本共同训练本地模型。理论上，我们分析了Re-Fed识别重要回放样本的能力，从而缓解灾难性遗忘问题。此外，实验表明，与现有最优方法相比，Re-Fed取得了具有竞争力的性能。