This research paper presents a part-of-speech (POS) annotated dataset and tagger tool for the low-resource Uzbek language. The dataset includes 12 tags, which were used to develop a rule-based POS-tagger tool. The corpus text used in the annotation process was made sure to be balanced over 20 different fields in order to ensure its representativeness. Uzbek being an agglutinative language so the most of the words in an Uzbek sentence are formed by adding suffixes. This nature of it makes the POS-tagging task difficult to find the stems of words and the right part-of-speech they belong to. The methodology proposed in this research is the stemming of the words with an affix/suffix stripping approach including database of the stem forms of the words in the Uzbek language. The tagger tool was tested on the annotated dataset and showed high accuracy in identifying and tagging parts of speech in Uzbek text. This newly presented dataset and tagger tool can be used for a variety of natural language processing tasks such as language modeling, machine translation, and text-to-speech synthesis. The presented dataset is the first of its kind to be made publicly available for Uzbek, and the POS-tagger tool created can also be used as a pivot to use as a base for other closely-related Turkic languages.
翻译:本研究论文为低资源语言乌兹别克语构建了一个词性标注数据集及标注工具。该数据集包含12个词性标签,并基于此开发了基于规则的词性标注工具。为保障语料代表性,标注过程中使用的语料文本确保覆盖20个不同领域。由于乌兹别克语属黏着语,其句子中多数词汇通过添加后缀构成,这一特性使得词干提取及正确词性判别成为词性标注任务的难点。本文提出的方法论采用词干提取技术,结合乌兹别克语词干形态数据库进行词缀剥离。该标注工具在已标注数据集上测试,展现出对乌兹别克语文本词性识别与标注的高准确性。这一新发布的数据集与标注工具可应用于语言建模、机器翻译及语音合成等多种自然语言处理任务。作为首个公开可用的乌兹别克语词性标注数据集,该工具亦可作为基础框架,为其他同源突厥语系语言的研究提供支撑。