Large Language Models (LLMs) and Text-To-Image (T2I) models have demonstrated the ability to generate compelling text and visual stories. However, their outputs are predominantly aligned with the sensibilities of the Global North, often resulting in an outsider's gaze on other cultures. As a result, non-Western communities have to put extra effort into generating culturally specific stories. To address this challenge, we developed a visual storytelling pipeline called KAHANI that generates culturally grounded visual stories for non-Western cultures. Our pipeline leverages off-the-shelf models GPT-4 Turbo and Stable Diffusion XL (SDXL). By using Chain of Thought (CoT) and T2I prompting techniques, we capture the cultural context from user's prompt and generate vivid descriptions of the characters and scene compositions. To evaluate the effectiveness of KAHANI, we conducted a comparative user study with ChatGPT-4 (with DALL-E3) in which participants from different regions of India compared the cultural relevance of stories generated by the two tools. Results from the qualitative and quantitative analysis performed on the user study showed that KAHANI was able to capture and incorporate more Culturally Specific Items (CSIs) compared to ChatGPT-4. In terms of both its cultural competence and visual story generation quality, our pipeline outperformed ChatGPT-4 in 27 out of the 36 comparisons.

翻译：大型语言模型（LLM）与文本到图像（T2I）模型已展现出生成引人入胜的文本与视觉故事的能力。然而，其输出内容主要与全球北方地区的文化感知对齐，往往导致对其他文化采用"外来者凝视"的视角。因此，非西方社群需要付出额外努力才能生成具有文化特异性的故事。为应对这一挑战，我们开发了名为KAHANI的视觉叙事生成流程，可为非西方文化生成基于文化根基的视觉故事。该流程利用现成模型GPT-4 Turbo与Stable Diffusion XL（SDXL），通过思维链（CoT）和T2I提示技术，从用户提示中捕捉文化语境，并生成对角色与场景构图的生动描述。为评估KAHANI的有效性，我们与ChatGPT-4（搭载DALL-E3）进行了对比用户研究，来自印度不同地区的参与者比较了两款工具生成故事的文化相关性。对用户研究进行的定性与定量分析结果表明，与ChatGPT-4相比，KAHANI能够捕捉并融入更多文化特定要素（CSI）。在文化理解能力与视觉故事生成质量两方面，我们的流程在36组对比中有27组表现优于ChatGPT-4。