Word sense disambiguation (WSD), which aims to determine an appropriate sense for a target word given its context, is crucial for natural language understanding. Existing supervised methods treat WSD as a classification task and have achieved remarkable performance. However, they ignore uncertainty estimation (UE) in the real-world setting, where the data is always noisy and out of distribution. This paper extensively studies UE on the benchmark designed for WSD. Specifically, we first compare four uncertainty scores for a state-of-the-art WSD model and verify that the conventional predictive probabilities obtained at the end of the model are inadequate to quantify uncertainty. Then, we examine the capability of capturing data and model uncertainties by the model with the selected UE score on well-designed test scenarios and discover that the model reflects data uncertainty satisfactorily but underestimates model uncertainty. Furthermore, we explore numerous lexical properties that intrinsically affect data uncertainty and provide a detailed analysis of four critical aspects: the syntactic category, morphology, sense granularity, and semantic relations.

翻译：词义消歧（WSD）旨在根据上下文确定目标词的恰当义项，是自然语言理解的关键任务。现有监督方法将WSD视为分类任务并取得了显著性能，但忽略了现实场景中数据存在噪声和分布外情况时的不确定性估计（UE）。本文在专为WSD设计的基准上系统研究了UE问题。具体而言，我们首先针对最先进的WSD模型比较了四种不确定性分数，验证了模型末端获得的传统预测概率不足以量化不确定性。随后，我们通过精心设计的测试场景，考察了采用选定UE分数的模型捕获数据不确定性和模型不确定性的能力，发现模型能较好地反映数据不确定性，但低估了模型不确定性。此外，我们探索了诸多内在地影响数据不确定性的词汇属性，并对四个关键方面进行了详细分析：句法类别、形态学、义项粒度和语义关系。