Image Captioning generates descriptive sentences from images using Vision-Language Pre-trained models (VLPs) such as BLIP, which has improved greatly. However, current methods lack the generation of detailed descriptive captions for the cultural elements depicted in the images, such as the traditional clothing worn by people from Asian cultural groups. In this paper, we propose a new framework, \textbf{Culturally-aware Image Captioning (CIC)}, that generates captions and describes cultural elements extracted from cultural visual elements in images representing cultures. Inspired by methods combining visual modality and Large Language Models (LLMs) through appropriate prompts, our framework (1) generates questions based on cultural categories from images, (2) extracts cultural visual elements from Visual Question Answering (VQA) using generated questions, and (3) generates culturally-aware captions using LLMs with the prompts. Our human evaluation conducted on 45 participants from 4 different cultural groups with a high understanding of the corresponding culture shows that our proposed framework generates more culturally descriptive captions when compared to the image captioning baseline based on VLPs. Our code and dataset will be made publicly available upon acceptance.

翻译：图像描述生成利用视觉-语言预训练模型（如BLIP）从图像生成描述性语句，近年来取得了显著进步。然而，现有方法缺乏对图像中文化元素（如亚洲文化群体人物的传统服饰）的详细描述性标注生成能力。本文提出一种新框架——**文化感知图像描述生成（CIC）**，该框架通过提取表征文化内涵的视觉元素生成描述语句与文化元素。受基于适当提示结合视觉模态与大语言模型（LLM）方法的启发，本框架通过以下步骤实现：（1）基于图像文化类别生成问题；（2）利用生成问题通过视觉问答（VQA）提取文化视觉元素；（3）结合提示词通过大语言模型生成文化感知描述。我们招募了来自4个不同文化群体且对相应文化有深度理解的45名参与者进行人工评估，结果表明，相较于基于视觉-语言预训练模型的图像描述基线方法，本框架能生成更具文化描述性的标注文本。论文被接收后，我们将公开代码与数据集。