This work investigates the impact of data augmentation on confidence calibration and uncertainty estimation in Named Entity Recognition (NER) tasks. For the future advance of NER in safety-critical fields like healthcare and finance, it is essential to achieve accurate predictions with calibrated confidence when applying Deep Neural Networks (DNNs), including Pre-trained Language Models (PLMs), as a real-world application. However, DNNs are prone to miscalibration, which limits their applicability. Moreover, existing methods for calibration and uncertainty estimation are computational expensive. Our investigation in NER found that data augmentation improves calibration and uncertainty in cross-genre and cross-lingual setting, especially in-domain setting. Furthermore, we showed that the calibration for NER tends to be more effective when the perplexity of the sentences generated by data augmentation is lower, and that increasing the size of the augmentation further improves calibration and uncertainty.

翻译：本研究探讨了数据增强对命名实体识别任务中置信度校准与不确定性估计的影响。为使命名实体识别在医疗、金融等安全关键领域取得未来进展，在将深度神经网络（包括预训练语言模型）应用于实际场景时，必须实现具有校准置信度的精准预测。然而，深度神经网络易出现校准错误，这限制了其适用性。此外，现有的校准与不确定性估计方法计算成本高昂。我们在命名实体识别中的研究发现，数据增强能提升跨领域与跨语言场景（尤其是领域内场景）的校准效果与不确定性度量。进一步研究表明，当数据增强生成的句子困惑度较低时，命名实体识别的校准往往更有效；且增加增强数据规模能进一步提升校准质量与不确定性估计效果。