Natural language processing for the Turkic language family, spoken by over 200 million people across Eurasia, remains fragmented, with most languages lacking unified tooling and resources. We present TurkicNLP, an open-source Python library providing a single, consistent NLP pipeline for Turkic languages across four script families: Latin, Cyrillic, Perso-Arabic, and Old Turkic Runic. The library covers tokenization, morphological analysis, part-of-speech tagging, dependency parsing, named entity recognition, bidirectional script transliteration, cross-lingual sentence embeddings, and machine translation through one language-agnostic API. A modular multi-backend architecture integrates rule-based finite-state transducers and neural models transparently, with automatic script detection and routing between script variants. Outputs follow the CoNLL-U standard for full interoperability and extension. Code and documentation are hosted at https://github.com/turkic-nlp/turkicnlp .
翻译:突厥语系由欧亚大陆超过2亿人使用,但其自然语言处理仍处于碎片化状态,多数语言缺乏统一的工具与资源支持。我们提出TurkicNLP——一个开源Python库,为拉丁、西里尔、波斯-阿拉伯及古突厥如尼文四大文字体系的突厥语言提供统一且一致的NLP流水线。该库通过语言无关的API实现词元化、形态分析、词性标注、依存句法分析、命名实体识别、双向文字转写、跨语言句子嵌入及机器翻译。模块化多后端架构透明集成了基于规则的有穷状态转换器与神经模型,并具备自动脚本检测与文字变体路由功能。输出采用CoNLL-U标准以确保完全互操作性与可扩展性。代码与文档托管于 https://github.com/turkic-nlp/turkicnlp 。