This paper presents the development process of a Vietnamese spoken language corpus for machine reading comprehension (MRC) tasks and provides insights into the challenges and opportunities associated with using real-world data for machine reading comprehension tasks. The existing MRC corpora in Vietnamese mainly focus on formal written documents such as Wikipedia articles, online newspapers, or textbooks. In contrast, the VlogQA consists of 10,076 question-answer pairs based on 1,230 transcript documents sourced from YouTube -- an extensive source of user-uploaded content, covering the topics of food and travel. By capturing the spoken language of native Vietnamese speakers in natural settings, an obscure corner overlooked in Vietnamese research, the corpus provides a valuable resource for future research in reading comprehension tasks for the Vietnamese language. Regarding performance evaluation, our deep-learning models achieved the highest F1 score of 75.34% on the test set, indicating significant progress in machine reading comprehension for Vietnamese spoken language data. In terms of EM, the highest score we accomplished is 53.97%, which reflects the challenge in processing spoken-based content and highlights the need for further improvement.
翻译:本文介绍了面向机器阅读理解(MRC)任务的越南语口语语料库开发过程,并深入探讨了利用真实世界数据开展机器阅读理解任务所面临的挑战与机遇。现有的越南语MRC语料库主要聚焦于正式书面文档,如维基百科文章、在线新闻或教科书。相比之下,VlogQA包含基于1,230份转录文档的10,076个问答对,这些文档源自YouTube——一个由用户生成内容构成的丰富来源,涵盖美食与旅行主题。通过捕捉越南语母语者在自然场景中的口语表达——这一在越南语研究中被忽视的领域,该语料库为未来越南语阅读理解任务研究提供了宝贵资源。在性能评估方面,我们的深度学习模型在测试集上取得了75.34%的最高F1分数,表明越南语口语数据的机器阅读理解取得了显著进展。在精确匹配(EM)指标上,我们取得的最佳分数为53.97%,这反映了处理口语内容的挑战性,并凸显了进一步改进的必要性。