Federated Named Entity Recognition (FNER) boosts model training within each local client by aggregating the model updates of decentralized local clients, without sharing their private data. However, existing FNER methods assume fixed entity types and local clients in advance, leading to their ineffectiveness in practical applications. In a more realistic scenario, local clients receive new entity types continuously, while new local clients collecting novel data may irregularly join the global FNER training. This challenging setup, referred to here as Federated Incremental NER, renders the global model suffering from heterogeneous forgetting of old entity types from both intra-client and inter-client perspectives. To overcome these challenges, we propose a Local-Global Forgetting Defense (LGFD) model. Specifically, to address intra-client forgetting, we develop a structural knowledge distillation loss to retain the latent space's feature structure and a pseudo-label-guided inter-type contrastive loss to enhance discriminative capability over different entity types, effectively preserving previously learned knowledge within local clients. To tackle inter-client forgetting, we propose a task switching monitor that can automatically identify new entity types under privacy protection and store the latest old global model for knowledge distillation and pseudo-labeling. Experiments demonstrate significant improvement of our LGFD model over comparison methods.
翻译:联邦命名实体识别(FNER)通过聚合分散的本地客户端的模型更新来增强每个本地客户端内的模型训练,而无需共享其私有数据。然而,现有的FNER方法预先假设固定的实体类型和本地客户端,导致其在现实应用中效果不佳。在更现实的场景中,本地客户端持续接收新的实体类型,同时收集新数据的新本地客户端可能不定期地加入全局FNER训练。这一具有挑战性的设置,在此称为联邦增量NER,使得全局模型从客户端内部和客户端间两个角度都遭受对旧实体类型的异构遗忘。为了克服这些挑战,我们提出了一个本地-全局遗忘防御(LGFD)模型。具体而言,为解决客户端内部遗忘,我们开发了一种结构化知识蒸馏损失以保留潜在空间的特征结构,以及一种伪标签引导的类间对比损失以增强对不同实体类型的判别能力,从而有效保留本地客户端内先前学到的知识。为解决客户端间遗忘,我们提出了一种任务切换监控器,它能够在隐私保护下自动识别新的实体类型,并存储最新的旧全局模型以用于知识蒸馏和伪标签生成。实验证明,我们的LGFD模型相较于对比方法有显著提升。