For named entity recognition (NER) in zero-resource languages, utilizing knowledge distillation methods to transfer language-independent knowledge from the rich-resource source languages to zero-resource languages is an effective means. Typically, these approaches adopt a teacher-student architecture, where the teacher network is trained in the source language, and the student network seeks to learn knowledge from the teacher network and is expected to perform well in the target language. Despite the impressive performance achieved by these methods, we argue that they have two limitations. Firstly, the teacher network fails to effectively learn language-independent knowledge shared across languages due to the differences in the feature distribution between the source and target languages. Secondly, the student network acquires all of its knowledge from the teacher network and ignores the learning of target language-specific knowledge. Undesirably, these limitations would hinder the model's performance in the target language. This paper proposes an unsupervised prototype knowledge distillation network (ProKD) to address these issues. Specifically, ProKD presents a contrastive learning-based prototype alignment method to achieve class feature alignment by adjusting the distance among prototypes in the source and target languages, boosting the teacher network's capacity to acquire language-independent knowledge. In addition, ProKD introduces a prototypical self-training method to learn the intrinsic structure of the language by retraining the student network on the target data using samples' distance information from prototypes, thereby enhancing the student network's ability to acquire language-specific knowledge. Extensive experiments on three benchmark cross-lingual NER datasets demonstrate the effectiveness of our approach.

翻译：针对零资源语言的命名实体识别（NER），利用知识蒸馏方法将知识从资源丰富的源语言迁移至零资源语言是一种有效手段。典型方法采用教师-学生架构：教师网络在源语言上训练，学生网络则试图从教师网络学习知识，并期望在目标语言上表现良好。尽管这些方法取得了显著性能，我们认为其存在两个局限：首先，由于源语言与目标语言特征分布差异，教师网络未能有效学习跨语言共享的语言无关知识；其次，学生网络完全从教师网络获取知识，忽略了目标语言特定知识的学习。这些局限将阻碍模型在目标语言上的表现。本文提出无监督原型知识蒸馏网络（ProKD）以解决上述问题。具体而言，ProKD提出基于对比学习的原型对齐方法，通过调整源语言与目标语言中原型间的距离实现类别特征对齐，从而增强教师网络获取语言无关知识的能力。此外，ProKD引入原型自训练方法，利用样本与原型间的距离信息在目标数据上重新训练学生网络，以学习语言内在结构，从而提升学生网络获取语言特定知识的能力。在三个基准跨语言NER数据集上的大量实验验证了本方法的有效性。