Visual Question Answering (VQA) is a complicated task that requires the capability of simultaneously processing natural language and images. Initially, this task was researched, focusing on methods to help machines understand objects and scene contexts in images. However, some text appearing in the image that carries explicit information about the full content of the image is not mentioned. Along with the continuous development of the AI era, there have been many studies on the reading comprehension ability of VQA models in the world. As a developing country, conditions are still limited, and this task is still open in Vietnam. Therefore, we introduce the first large-scale dataset in Vietnamese specializing in the ability to understand text appearing in images, we call it ViTextVQA (\textbf{Vi}etnamese \textbf{Text}-based \textbf{V}isual \textbf{Q}uestion \textbf{A}nswering dataset) which contains \textbf{over 16,000} images and \textbf{over 50,000} questions with answers. Through meticulous experiments with various state-of-the-art models, we uncover the significance of the order in which tokens in OCR text are processed and selected to formulate answers. This finding helped us significantly improve the performance of the baseline models on the ViTextVQA dataset. Our dataset is available at this \href{https://github.com/minhquan6203/ViTextVQA-Dataset}{link} for research purposes.
翻译:视觉问答是一项复杂的任务,需要同时具备处理自然语言和图像的能力。最初,该任务的研究重点在于帮助机器理解图像中的物体和场景上下文的方法。然而,图像中出现的一些承载图像完整内容显式信息的文本并未得到关注。随着人工智能时代的持续发展,全球范围内已涌现出大量关于视觉问答模型阅读理解能力的研究。作为一个条件仍有限的发展中国家,这一任务在越南仍处于待开发状态。因此,我们引入了首个专攻图像中文本理解能力的越南语大规模数据集,将其命名为ViTextVQA(越南语基于文本的视觉问答数据集),该数据集包含**超过16,000张**图像和**超过50,000个**带答案的问题。通过使用多种最先进模型进行细致的实验,我们揭示了OCR文本中词元处理顺序与答案构建之间关联的重要性。这一发现显著提升了基线模型在ViTextVQA数据集上的性能。我们在此\href{https://github.com/minhquan6203/ViTextVQA-Dataset}{链接}提供数据集供研究使用。