Incremental Named Entity Recognition (INER) involves the sequential learning of new entity types without accessing the training data of previously learned types. However, INER faces the challenge of catastrophic forgetting specific for incremental learning, further aggravated by background shift (i.e., old and future entity types are labeled as the non-entity type in the current task). To address these challenges, we propose a method called task Relation Distillation and Prototypical pseudo label (RDP) for INER. Specifically, to tackle catastrophic forgetting, we introduce a task relation distillation scheme that serves two purposes: 1) ensuring inter-task semantic consistency across different incremental learning tasks by minimizing inter-task relation distillation loss, and 2) enhancing the model's prediction confidence by minimizing intra-task self-entropy loss. Simultaneously, to mitigate background shift, we develop a prototypical pseudo label strategy that distinguishes old entity types from the current non-entity type using the old model. This strategy generates high-quality pseudo labels by measuring the distances between token embeddings and type-wise prototypes. We conducted extensive experiments on ten INER settings of three benchmark datasets (i.e., CoNLL2003, I2B2, and OntoNotes5). The results demonstrate that our method achieves significant improvements over the previous state-of-the-art methods, with an average increase of 6.08% in Micro F1 score and 7.71% in Macro F1 score.

翻译：增量命名实体识别（INER）涉及在不访问先前学习过的实体类型训练数据的情况下，顺序学习新的实体类型。然而，INER面临着增量学习特有的灾难性遗忘挑战，而背景偏移（即旧的和未来的实体类型在当前任务中被标记为非实体类型）进一步加剧了这一问题。为解决这些挑战，我们提出了一种名为任务关系蒸馏与原型伪标签（RDP）的方法用于INER。具体而言，为应对灾难性遗忘，我们引入了一种任务关系蒸馏方案，该方案有两个目的：1）通过最小化任务间关系蒸馏损失，确保不同增量学习任务之间的跨任务语义一致性；2）通过最小化任务内自熵损失，增强模型的预测置信度。同时，为缓解背景偏移问题，我们开发了一种原型伪标签策略，该策略利用旧模型区分旧实体类型与当前非实体类型。该策略通过测量词元嵌入与类型级原型之间的距离，生成高质量的伪标签。我们在三个基准数据集（即CoNLL2003、I2B2和OntoNotes5）的十个INER设置上进行了广泛实验。结果表明，我们的方法相比先前最先进方法取得了显著改进，Micro F1分数平均提升6.08%，Macro F1分数平均提升7.71%。