Word Sense Disambiguation (WSD) has been widely evaluated using the semantic frameworks of WordNet, BabelNet, and the Oxford Dictionary of English. However, for the UCREL Semantic Analysis System (USAS) framework, no open extensive evaluation has been performed beyond lexical coverage or single language evaluation. In this work, we perform the largest semantic tagging evaluation of the rule based system that uses the lexical resources in the USAS framework covering five different languages using four existing datasets and one novel Chinese dataset. We create a new silver labelled English dataset, to overcome the lack of manually tagged training data, that we train and evaluate various mono and multilingual neural models in both mono and cross-lingual evaluation setups with comparisons to their rule based counterparts, and show how a rule based system can be enhanced with a neural network model. The resulting neural network models, including the data they were trained on, the Chinese evaluation dataset, and all of the code have been released as open resources.

翻译：词义消歧（WSD）长期以来广泛采用WordNet、BabelNet及《牛津英语词典》等语义框架进行评估。然而，对于UCREL语义分析系统（USAS）框架，除词汇覆盖度或单一语言评估外，尚未进行公开的大规模评估。本研究利用USAS框架中的词汇资源，通过四个现有数据集和一个新构建的中文数据集，对覆盖五种语言的基于规则的语义标注系统进行了迄今最大规模的评估。为克服人工标注训练数据的缺乏，我们创建了新的银标注英文数据集，并在此基础上训练和评估了多种单语及多语言神经模型，同时在单语与跨语言评估设置中与基于规则的模型进行对比，展示了如何通过神经网络模型增强基于规则的系统。最终发布的开放资源包括训练所得的神经网络模型、其训练数据、中文评估数据集以及全部代码。