Image Captioning generates descriptive sentences from images using Vision-Language Pre-trained models (VLPs) such as BLIP, which has improved greatly. However, current methods lack the generation of detailed descriptive captions for the cultural elements depicted in the images, such as the traditional clothing worn by people from Asian cultural groups. In this paper, we propose a new framework, \textbf{Culturally-aware Image Captioning (CIC)}, that generates captions and describes cultural elements extracted from cultural visual elements in images representing cultures. Inspired by methods combining visual modality and Large Language Models (LLMs) through appropriate prompts, our framework (1) generates questions based on cultural categories from images, (2) extracts cultural visual elements from Visual Question Answering (VQA) using generated questions, and (3) generates culturally-aware captions using LLMs with the prompts. Our human evaluation conducted on 45 participants from 4 different cultural groups with a high understanding of the corresponding culture shows that our proposed framework generates more culturally descriptive captions when compared to the image captioning baseline based on VLPs. Our code and dataset will be made publicly available upon acceptance.

翻译：图像描述生成利用视觉-语言预训练模型（如BLIP）生成图像的描述性句子，该类技术已取得显著进展。然而，现有方法缺乏对图像中文化元素（例如亚洲文化群体人物的传统服饰）的详细描述性标注生成能力。本文提出新框架——**文化感知图像描述生成（CIC）**，该框架能够为表现不同文化的图像生成描述，并提取其中蕴含的文化视觉元素。受通过适当的提示词将视觉模态与大语言模型（LLMs）相结合的方法启发，本框架：（1）基于图像中的文化类别生成问题，（2）通过视觉问答（VQA）利用生成的问题提取文化视觉元素，（3）使用提示词驱动LLMs生成文化感知描述。我们招募了来自4个不同文化群体、对相应文化具有高理解度的45名参与者的人工评估结果表明，与基于视觉-语言预训练模型的图像描述基线方法相比，所提框架能生成更具文化描述性的标注。代码与数据集将在论文接收后公开。