Diffusion-based generative models have significantly advanced text-to-image generation but encounter challenges when processing lengthy and intricate text prompts describing complex scenes with multiple objects. While excelling in generating images from short, single-object descriptions, these models often struggle to faithfully capture all the nuanced details within longer and more elaborate textual inputs. In response, we present a novel approach leveraging Large Language Models (LLMs) to extract critical components from text prompts, including bounding box coordinates for foreground objects, detailed textual descriptions for individual objects, and a succinct background context. These components form the foundation of our layout-to-image generation model, which operates in two phases. The initial Global Scene Generation utilizes object layouts and background context to create an initial scene but often falls short in faithfully representing object characteristics as specified in the prompts. To address this limitation, we introduce an Iterative Refinement Scheme that iteratively evaluates and refines box-level content to align them with their textual descriptions, recomposing objects as needed to ensure consistency. Our evaluation on complex prompts featuring multiple objects demonstrates a substantial improvement in recall compared to baseline diffusion models. This is further validated by a user study, underscoring the efficacy of our approach in generating coherent and detailed scenes from intricate textual inputs.

翻译：基于扩散的生成模型在文本到图像生成方面取得了显著进展，但在处理描述复杂场景（包含多个物体）的长篇复杂文本提示时仍面临挑战。尽管这类模型能出色地生成简短单物体描述对应的图像，但在捕捉冗长文本输入中的细微细节方面往往力不从心。为此，我们提出了一种利用大语言模型（LLM）的新方法，从文本提示中提取关键组件，包括前景物体的边界框坐标、单个物体的详细文本描述以及简洁的背景上下文。这些组件构成了我们布局到图像生成模型的基础，该模型分两个阶段运行。初始的全局场景生成阶段利用物体布局和背景上下文创建初步场景，但通常难以忠实呈现提示中指定的物体特征。为解决这一局限，我们引入了迭代优化方案，通过反复评估和细化逐框内容使其与文本描述对齐，并在必要时重组物体以确保一致性。在包含多物体的复杂提示上的评估表明，与基线扩散模型相比，我们的方法在召回率上实现了显著提升。这进一步通过用户研究得到验证，充分证明了我们的方法在从复杂文本输入生成连贯且细节丰富场景方面的有效性。