This paper introduces UQA, a novel dataset for question answering and text comprehension in Urdu, a low-resource language with over 70 million native speakers. UQA is generated by translating the Stanford Question Answering Dataset (SQuAD2.0), a large-scale English QA dataset, using a technique called EATS (Enclose to Anchor, Translate, Seek), which preserves the answer spans in the translated context paragraphs. The paper describes the process of selecting and evaluating the best translation model among two candidates: Google Translator and Seamless M4T. The paper also benchmarks several state-of-the-art multilingual QA models on UQA, including mBERT, XLM-RoBERTa, and mT5, and reports promising results. For XLM-RoBERTa-XL, we have an F1 score of 85.99 and 74.56 EM. UQA is a valuable resource for developing and testing multilingual NLP systems for Urdu and for enhancing the cross-lingual transferability of existing models. Further, the paper demonstrates the effectiveness of EATS for creating high-quality datasets for other languages and domains. The UQA dataset and the code are publicly available at www.github.com/sameearif/UQA.
翻译:本文介绍了UQA,一个面向乌尔都语(一种拥有超过7000万母语者的低资源语言)问答与文本理解的新型数据集。UQA是通过采用名为EATS(Enclose to Anchor, Translate, Seek)的技术翻译斯坦福问答数据集(SQuAD2.0,一个大规模英文问答数据集)生成的,该技术能够在翻译后的上下文段落中保留答案跨度。本文描述了在Google Translator和Seamless M4T两个候选翻译模型中选取并评估最优模型的过程。同时,本文在UQA上对多个最先进的多语言问答模型(包括mBERT、XLM-RoBERTa和mT5)进行了基准测试,并报告了具有前景的结果。对于XLM-RoBERTa-XL,其F1分数为85.99,精确匹配(EM)分数为74.56。UQA为开发和测试面向乌尔都语的多语言NLP系统,以及增强现有模型的跨语言迁移能力提供了宝贵资源。此外,本文展示了EATS在创建其他语言和领域的高质量数据集方面的有效性。UQA数据集及代码已在www.github.com/sameearif/UQA公开。