This paper focuses on developing translation models and related applications for 36 Indian languages, including Assamese, Awadhi, Bengali, Bhojpuri, Braj, Bodo, Dogri, English, Konkani, Gondi, Gujarati, Hindi, Hinglish, Ho, Kannada, Kangri, Kashmiri (Arabic and Devanagari), Khasi, Mizo, Magahi, Maithili, Malayalam, Marathi, Manipuri (Bengali and Meitei), Nepali, Oriya, Punjabi, Sanskrit, Santali, Sinhala, Sindhi (Arabic and Devanagari), Tamil, Tulu, Telugu, and Urdu. Achieving this requires parallel and other types of corpora for all 36 * 36 language pairs, addressing challenges like script variations, phonetic differences, and syntactic diversity. For instance, languages like Kashmiri and Sindhi, which use multiple scripts, demand script normalization for alignment, while low-resource languages such as Khasi and Santali require synthetic data augmentation to ensure sufficient coverage and quality. To address these challenges, this work proposes strategies for corpus creation by leveraging existing resources, developing parallel datasets, generating domain-specific corpora, and utilizing synthetic data techniques. Additionally, it evaluates machine translation across various dimensions, including standard and discourse-level translation, domain-specific translation, reference-based and reference-free evaluation, error analysis, and automatic post-editing. By integrating these elements, the study establishes a comprehensive framework to improve machine translation quality and enable better cross-lingual communication in India's linguistically diverse ecosystem.
翻译:本文聚焦于为36种印度语言开发翻译模型及相关应用,涵盖阿萨姆语、阿瓦德语、孟加拉语、博杰普尔语、布拉吉语、博多语、多格里语、英语、孔卡尼语、贡德语、古吉拉特语、印地语、印英混合语、霍语、卡纳达语、坎格里语、克什米尔语(阿拉伯文与天城文变体)、卡西语、米佐语、马加希语、迈蒂利语、马拉雅拉姆语、马拉地语、曼尼普尔语(孟加拉文与梅泰文变体)、尼泊尔语、奥里亚语、旁遮普语、梵语、桑塔利语、僧伽罗语、信德语(阿拉伯文与天城文变体)、泰米尔语、图鲁语、泰卢固语及乌尔都语。实现此目标需要为全部36×36种语言对构建平行语料库及其他类型语料资源,并应对文字变体、语音差异及句法多样性等挑战。例如,克什米尔语和信德语等使用多种文字的语言需进行文字规范化以实现对齐,而卡西语和桑塔利语等低资源语言则需通过合成数据增强来保证足够的覆盖范围与质量。为应对这些挑战,本研究提出通过整合现有资源、开发平行数据集、生成领域专用语料库以及运用合成数据技术等语料构建策略。此外,研究从多维度评估机器翻译性能,包括标准翻译与篇章级翻译、领域专用翻译、基于参考与非参考的评估、错误分析及自动译后编辑。通过整合这些要素,本研究构建了一个综合框架,旨在提升机器翻译质量,促进印度多语言生态系统中更高效的跨语言交流。