Adversarial evaluations of language models typically focus on English alone. In this paper, we performed a multilingual evaluation of Named Entity Recognition (NER) in terms of its robustness to small perturbations in the input. Our results showed the NER models we explored across three languages (English, German and Hindi) are not very robust to such changes, as indicated by the fluctuations in the overall F1 score as well as in a more fine-grained evaluation. With that knowledge, we further explored whether it is possible to improve the existing NER models using a part of the generated adversarial data sets as augmented training data to train a new NER model or as fine-tuning data to adapt an existing NER model. Our results showed that both these approaches improve performance on the original as well as adversarial test sets. While there is no significant difference between the two approaches for English, re-training is significantly better than fine-tuning for German and Hindi.

翻译：针对语言模型的对抗性评估通常仅聚焦于英语。本文对命名实体识别（NER）在输入微小扰动下的鲁棒性进行了多语言评估。结果表明，我们针对三种语言（英语、德语和印地语）所探索的NER模型对这类变化并不十分鲁棒——整体F1分数及更细粒度的评估结果均呈现波动。基于这一发现，我们进一步探究了利用部分生成的对抗性数据集作为增强训练数据来训练新NER模型，或作为微调数据来适配现有NER模型，从而改进现有NER模型的可能性。结果显示，两种方法均能提升原始测试集和对抗测试集的性能。尽管英语中两种方法无显著差异，但对于德语和印地语，重新训练的效果显著优于微调。