Humor is a commonly used and intricate human language in daily life. Humor generation, especially in multi-modal scenarios, is a challenging task for large language models (LLMs), which is typically as funny caption generation for images, requiring visual understanding, humor reasoning, creative imagination, and so on. Existing LLM-based approaches rely on reasoning chains or self-improvement, which suffer from limited creativity and interpretability. To address these bottlenecks, we develop a novel LLM-based humor generation mechanism based on a fundamental humor theory, GTVH. To produce funny and script-opposite captions, we introduce a humor-theory-driven multi-role LLM collaboration framework augmented with humor retrieval (HOMER). The framework consists of three LLM-based roles: (1) conflicting-script extractor that grounds humor in key script oppositions, forming the basis of caption generation; (2) retrieval-augmented hierarchical imaginator that identifies key humor targets and expands the creative space of them through diverse associations structured as imagination trees; and (3) caption generator that produces funny and diverse captions conditioned on the obtained knowledge. Extensive experiments on two New Yorker Cartoon benchmarking datasets show that HOMER outperforms state-of-the-art baselines and powerful LLM reasoning strategies on multi-modal humor captioning.

翻译：幽默是日常生活中一种常用且复杂的人类语言。幽默生成，尤其是在多模态场景下，对于大型语言模型而言是一项具有挑战性的任务，通常体现为图像生成幽默字幕，这需要视觉理解、幽默推理、创造性想象等多方面能力。现有基于大型语言模型的方法依赖于推理链或自我改进，存在创造力有限和可解释性不足的问题。为应对这些瓶颈，我们基于基础幽默理论GTVH，开发了一种新颖的基于大型语言模型的幽默生成机制。为生成有趣且脚本对立的字幕，我们引入了一种由幽默理论驱动的、结合幽默检索增强的多角色大型语言模型协作框架。该框架包含三个基于大型语言模型的角色：(1) 冲突脚本提取器，将幽默锚定于关键脚本对立，构成字幕生成的基础；(2) 检索增强的分层想象器，识别关键幽默目标，并通过构建为想象树的多样化联想扩展其创意空间；(3) 字幕生成器，基于所获知识生成有趣且多样化的字幕。在两个《纽约客》漫画基准数据集上的大量实验表明，HOMER在多模态幽默字幕生成任务上优于最先进的基线模型和强大的大型语言模型推理策略。