Modern transformer-based neural architectures yield impressive results in nearly every NLP task and Word Sense Disambiguation, the problem of discerning the correct sense of a word in a given context, is no exception. State-of-the-art approaches in WSD today leverage lexical information along with pre-trained embeddings from these models to achieve results comparable to human inter-annotator agreement on standard evaluation benchmarks. In the same vein, we experiment with several strategies to optimize bi-encoders for this specific task and propose alternative methods of presenting lexical information to our model. Through our multi-stage pre-training and fine-tuning pipeline we further the state of the art in Word Sense Disambiguation.

翻译：现代以变压器为基础的神经结构在几乎每一个NLP任务和Word Sense Disfendation(在特定背景下辨别一个词的正确感的问题)中都取得了令人印象深刻的成果。今天,WSD最先进的方法利用了这些模型的词汇信息以及经过预先培训的嵌入,取得了与人类之间关于标准评价基准的协议相类似的成果。 同样,我们试验了几项战略,优化这一具体任务的双编码器,并提出了向模型展示词汇信息的替代方法。通过多阶段培训前和微调管道,我们进一步提升了Word Sense Disbugication中的艺术状态。