As Federated Learning (FL) has gained increasing attention, it has become widely acknowledged that straightforwardly applying stochastic gradient descent (SGD) on the overall framework when learning over a sequence of tasks results in the phenomenon known as ``catastrophic forgetting''. Consequently, much FL research has centered on devising federated increasing learning methods to alleviate forgetting while augmenting knowledge. On the other hand, forgetting is not always detrimental. The selective amnesia, also known as federated unlearning, which entails the elimination of specific knowledge, can address privacy concerns and create additional ``space'' for acquiring new knowledge. However, there is a scarcity of extensive surveys that encompass recent advancements and provide a thorough examination of this issue. In this manuscript, we present an extensive survey on the topic of knowledge editing (augmentation/removal) in Federated Learning, with the goal of summarizing the state-of-the-art research and expanding the perspective for various domains. Initially, we introduce an integrated paradigm, referred to as Federated Editable Learning (FEL), by reevaluating the entire lifecycle of FL. Secondly, we provide a comprehensive overview of existing methods, evaluate their position within the proposed paradigm, and emphasize the current challenges they face. Lastly, we explore potential avenues for future research and identify unresolved issues.
翻译:随着联邦学习(FL)日益受到关注,人们普遍认识到,在连续任务学习过程中对整体框架直接应用随机梯度下降(SGD)会导致所谓的“灾难性遗忘”现象。为此,大量FL研究聚焦于设计联邦增量学习方法,以在增强知识的同时缓解遗忘。另一方面,遗忘并非总是有害的。选择性遗忘(亦称联邦遗忘学习)涉及特定知识的消除,既能解决隐私问题,又能为获取新知识创造额外“空间”。然而,目前尚缺乏涵盖最新进展并对此问题进行深入分析的综合综述。本文对联邦学习中的知识编辑(增强/删除)进行了全面综述,旨在总结现有前沿研究并拓展多领域视角。首先,我们通过重新审视FL的完整生命周期,提出了一种集成范式——联邦可编辑学习(FEL)。其次,我们系统梳理了现有方法,评估其在所提范式中的定位,并着重分析了当前面临的挑战。最后,我们探讨了未来研究的潜在方向,并指出尚未解决的问题。