Recent advancements in text-to-image models, particularly diffusion models, have shown significant promise. However, compositional text-to-image models frequently encounter difficulties in generating high-quality images that accurately align with input texts describing multiple objects, variable attributes, and intricate spatial relationships. To address this limitation, we employ large vision-language models (LVLMs) for multi-dimensional assessment of the alignment between generated images and their corresponding input texts. Utilizing this assessment, we fine-tune the diffusion model to enhance its alignment capabilities. During the inference phase, an initial image is produced using the fine-tuned diffusion model. The LVLM is then employed to pinpoint areas of misalignment in the initial image, which are subsequently corrected using the image editing algorithm until no further misalignments are detected by the LVLM. The resultant image is consequently more closely aligned with the input text. Our experimental results validate that the proposed methodology significantly improves text-image alignment in compositional image generation, particularly with respect to object number, attribute binding, spatial relationships, and aesthetic quality.

翻译：近期，文本到图像模型（尤其是扩散模型）取得了显著进展。然而，组合式文本到图像模型在生成高质量图像时，常常难以准确匹配描述多个物体、可变属性和复杂空间关系的输入文本。为克服这一局限，我们采用大型视觉语言模型（LVLMs）对生成图像与对应输入文本之间的对齐程度进行多维度评估。基于此评估，我们对扩散模型进行微调以增强其对齐能力。在推理阶段，首先利用微调后的扩散模型生成初始图像。随后，借助LVLMs识别初始图像中的不对齐区域，并通过图像编辑算法进行修正，直至LVLMs未检测到进一步的不对齐现象。最终生成的图像与输入文本的匹配度显著提升。我们的实验结果验证，所提出的方法在组合式图像生成中能显著改善文本-图像对齐效果，尤其在物体数量、属性绑定、空间关系以及美学质量方面表现突出。