This research paper presents a part-of-speech (POS) annotated dataset and tagger tool for the low-resource Uzbek language. The dataset includes 12 tags, which were used to develop a rule-based POS-tagger tool. The corpus text used in the annotation process was made sure to be balanced over 20 different fields in order to ensure its representativeness. Uzbek being an agglutinative language so the most of the words in an Uzbek sentence are formed by adding suffixes. This nature of it makes the POS-tagging task difficult to find the stems of words and the right part-of-speech they belong to. The methodology proposed in this research is the stemming of the words with an affix/suffix stripping approach including database of the stem forms of the words in the Uzbek language. The tagger tool was tested on the annotated dataset and showed high accuracy in identifying and tagging parts of speech in Uzbek text. This newly presented dataset and tagger tool can be used for a variety of natural language processing tasks such as language modeling, machine translation, and text-to-speech synthesis. The presented dataset is the first of its kind to be made publicly available for Uzbek, and the POS-tagger tool created can also be used as a pivot to use as a base for other closely-related Turkic languages.
翻译:本研究论文介绍了一个面向低资源乌兹别克语的词性标注数据集及标注工具。该数据集包含12个标注标签,并基于这些标签开发了一个规则驱动的词性标注工具。为保障语料代表性,标注过程中使用的语料库文本确保覆盖20个不同领域的均衡分布。作为黏着语,乌兹别克语句子中的大部分词汇通过添加后缀构成,这种语言特性使得词干提取和正确词性归属成为词性标注任务的难点。本文提出的方法论采用基于词缀/后缀剥离的词干提取方案,并包含乌兹别克语词干形态数据库。该标注工具在已标注数据集上进行了测试,在乌兹别克语文本词性识别与标注方面展现了高准确率。新提出的数据集和标注工具可广泛应用于语言建模、机器翻译及文本转语音等自然语言处理任务。作为首个公开可用的乌兹别克语词性标注数据集,其标注工具还可作为基础框架,为其他同源性突厥语系语言的研究提供支撑。