Image captioning requires numerous annotated image-text pairs, resulting in substantial annotation costs. Recently, large models (e.g. diffusion models and large language models) have excelled in producing high-quality images and text. This potential can be harnessed to create synthetic image-text pairs for training captioning models. Synthetic data can improve cost and time efficiency in data collection, allow for customization to specific domains, bootstrap generalization capability for zero-shot performance, and circumvent privacy concerns associated with real-world data. However, existing methods struggle to attain satisfactory performance solely through synthetic data. We identify the issue as generated images from simple descriptions mostly capture a solitary perspective with limited context, failing to align with the intricate scenes prevalent in real-world imagery. To tackle this, we present an innovative pipeline that introduces multi-context data generation. Beginning with an initial text corpus, our approach employs a large language model to extract multiple sentences portraying the same scene from diverse viewpoints. These sentences are then condensed into a single sentence with multiple contexts. Subsequently, we generate intricate images using the condensed captions through diffusion models. Our model is exclusively trained on synthetic image-text pairs crafted through this process. The effectiveness of our pipeline is validated through experimental results in both the in-domain and cross-domain settings, where it achieves state-of-the-art performance on well-known datasets such as MSCOCO, Flickr30k, and NoCaps.

翻译：图像描述生成需要大量带有标注的图像-文本对，导致高昂的标注成本。近年来，大型模型（如扩散模型和大语言模型）在生成高质量图像和文本方面表现卓越。这一潜力可被利用来创建用于训练描述模型的合成图像-文本对。合成数据能够提高数据收集的性价比与时间效率，支持针对特定领域的定制化需求，增强零样本场景下的泛化能力，并规避真实数据涉及的隐私问题。然而，现有方法仅依赖合成数据难以取得令人满意的性能。我们发现其问题在于：基于简单描述生成的图像大多仅呈现单一视角的有限情境，难以匹配真实图像中普遍存在的复杂场景。为此，我们提出一种创新流程，引入多情境数据生成方法。该方法首先以初始文本语料为起点，利用大语言模型从不同视角提取描述同一场景的多条语句，随后将这些语句压缩为包含多重情境的单一描述。接着，我们通过扩散模型利用压缩后的描述生成复杂图像。我们的模型仅通过此流程生成的合成图像-文本对进行训练。在域内与跨域场景下的实验结果验证了该流程的有效性，其在MSCOCO、Flickr30k和NoCaps等知名数据集上均取得了最先进的性能。