The primary focus of this thesis is to make Sanskrit manuscripts more accessible to the end-users through natural language technologies. The morphological richness, compounding, free word orderliness, and low-resource nature of Sanskrit pose significant challenges for developing deep learning solutions. We identify four fundamental tasks, which are crucial for developing a robust NLP technology for Sanskrit: word segmentation, dependency parsing, compound type identification, and poetry analysis. The first task, Sanskrit Word Segmentation (SWS), is a fundamental text processing task for any other downstream applications. However, it is challenging due to the sandhi phenomenon that modifies characters at word boundaries. Similarly, the existing dependency parsing approaches struggle with morphologically rich and low-resource languages like Sanskrit. Compound type identification is also challenging for Sanskrit due to the context-sensitive semantic relation between components. All these challenges result in sub-optimal performance in NLP applications like question answering and machine translation. Finally, Sanskrit poetry has not been extensively studied in computational linguistics. While addressing these challenges, this thesis makes various contributions: (1) The thesis proposes linguistically-informed neural architectures for these tasks. (2) We showcase the interpretability and multilingual extension of the proposed systems. (3) Our proposed systems report state-of-the-art performance. (4) Finally, we present a neural toolkit named SanskritShala, a web-based application that provides real-time analysis of input for various NLP tasks. Overall, this thesis contributes to making Sanskrit manuscripts more accessible by developing robust NLP technology and releasing various resources, datasets, and web-based toolkit.
翻译:本论文的主要目标是通过自然语言技术使梵语文献更易于终端用户使用。梵语的形态丰富性、复合词结构、自由语序以及低资源特性为开发深度学习解决方案带来了重大挑战。我们确定了四个对构建稳健梵语NLP技术至关重要的基础任务:分词、依存句法分析、复合词类型识别与诗歌分析。第一个任务“梵语分词”是其他下游应用的基础文本处理任务,但由于连音现象会改变词边界字符,该任务极具挑战性。类似地,现有的依存句法分析方法难以处理像梵语这样形态丰富且资源匮乏的语言。复合词类型识别同样因成分间依赖语境的语义关系而面临困难。这些挑战导致问答系统与机器翻译等NLP应用性能欠佳。此外,梵语诗歌在计算语言学领域尚未得到广泛研究。针对这些挑战,本论文做出以下贡献:(1)提出了基于语言知识的神经架构以处理上述任务;(2)展示了所提系统的可解释性与多语言扩展能力;(3)所提系统达到了最先进性能;(4)最终呈现了命名为SanskritShala的神经工具包——一个为多种NLP任务提供输入实时分析的网络应用。总体而言,本论文通过开发稳健NLP技术、发布多种资源、数据集与网络工具包,为提升梵语文献的可访问性做出了贡献。