In Federated Learning (FL), the data in each client is typically assumed fixed or static. However, data often comes in an incremental manner in real-world applications, where the data domain may increase dynamically. In this work, we study catastrophic forgetting with data heterogeneity in Federated Incremental Learning (FIL) scenarios where edge clients may lack enough storage space to retain full data. We propose to employ a simple, generic framework for FIL named Re-Fed, which can coordinate each client to cache important samples for replay. More specifically, when a new task arrives, each client first caches selected previous samples based on their global and local importance. Then, the client trains the local model with both the cached samples and the samples from the new task. Theoretically, we analyze the ability of Re-Fed to discover important samples for replay thus alleviating the catastrophic forgetting problem. Moreover, we empirically show that Re-Fed achieves competitive performance compared to state-of-the-art methods.

翻译：在联邦学习（FL）中，通常假设每个客户端的数据是固定或静态的。然而，在实际应用中，数据往往以增量方式到来，数据域可能动态增加。在这项工作中，我们研究了联邦增量学习（FIL）场景下，在数据异构性存在且边缘客户端可能缺乏足够存储空间来保留完整数据时的灾难性遗忘问题。我们提出采用一个简单、通用的FIL框架，名为Re-Fed，该框架可以协调每个客户端缓存重要样本用于回放。具体而言，当新任务到达时，每个客户端首先根据样本的全局和局部重要性缓存选定的先前样本。然后，客户端使用缓存样本和新任务的样本共同训练本地模型。理论上，我们分析了Re-Fed发现重要样本用于回放从而缓解灾难性遗忘问题的能力。此外，我们通过实验证明，与最先进的方法相比，Re-Fed取得了具有竞争力的性能。