Image Captioning is one of the vision-language tasks that still interest the research community worldwide in the 2020s. MS-COCO Caption benchmark is commonly used to evaluate the performance of advanced captioning models, although it was published in 2015. Recent captioning models trained on the MS-COCO Caption dataset only have good performance in language patterns of English; they do not have such good performance in contexts captured in Vietnam or fluently caption images using Vietnamese. To contribute to the low-resources research community as in Vietnam, we introduce a novel image captioning dataset in Vietnamese, the Open-domain Vietnamese Image Captioning dataset (UIT-OpenViIC). The introduced dataset includes complex scenes captured in Vietnam and manually annotated by Vietnamese under strict rules and supervision. In this paper, we present in more detail the dataset creation process. From preliminary analysis, we show that our dataset is challenging to recent state-of-the-art (SOTA) Transformer-based baselines, which performed well on the MS COCO dataset. Then, the modest results prove that UIT-OpenViIC has room to grow, which can be one of the standard benchmarks in Vietnamese for the research community to evaluate their captioning models. Furthermore, we present a CAMO approach that effectively enhances the image representation ability by a multi-level encoder output fusion mechanism, which helps improve the quality of generated captions compared to previous captioning models.

翻译：图像描述是2020年代仍受全球研究界关注的视觉-语言任务之一。尽管MS-COCO Caption基准数据集已于2015年发布，但至今仍被广泛用于评估先进描述模型的性能。然而，当前基于MS-COCO Caption数据集训练的先进描述模型仅在英语语言模式下表现良好，在越南场景中拍摄的图片上下文以及流畅生成越南语描述方面均存在不足。为助力越南等低资源语言研究社区，我们提出了一个全新的越南语图像描述数据集——开放域越南语图像描述数据集（UIT-OpenViIC）。该数据集包含越南场景中复杂场景的图像，并由越南语标注人员根据严格规则与监督进行人工标注。本文详细阐述了数据集的构建流程。初步分析表明，该数据集对近期在MS COCO数据集上表现优异的基于Transformer的先进基线模型具有挑战性。实验所得的有限结果证明UIT-OpenViIC数据集具有发展潜力，可成为研究社区评估越南语描述模型的标准基准之一。此外，我们提出了一种CAMO方法，通过多层编码器输出融合机制有效增强图像表征能力，相较于先前描述模型能显著提升生成描述的质量。