Recent advancements in image generation have achieved impressive results in producing high-quality images. However, existing image generation models still generally struggle with a spatial reasoning dilemma, lacking the ability to accurately capture fine-grained spatial relationships from the prompt and correctly generate scenes with structural integrity. To mitigate this dilemma, we propose RL-RIG, a Reinforcement Learning framework for Reflection-based Image Generation. Our architecture comprises four primary components: Diffuser, Checker, Actor, and Inverse Diffuser, following a Generate-Reflect-Edit paradigm to spark the Chain of Thought reasoning ability in image generation for addressing the dilemma. To equip the model with better intuition over generation trajectories, we further develop Reflection-GRPO to train the VLM Actor for edit prompts and the Image Editor for better image quality under a given prompt, respectively. Unlike traditional approaches that solely produce visually stunning yet structurally unreasonable content, our evaluation metrics prioritize spatial accuracy, utilizing Scene Graph IoU and employing a VLM-as-a-Judge strategy to assess the spatial consistency of generated images on LAION-SG dataset. Experimental results show that RL-RIG outperforms existing state-of-the-art open-source models by up to 11% in terms of controllable and precise spatial reasoning in image generation.

翻译：近年来，图像生成领域取得了显著进展，能够生成高质量图像。然而，现有的图像生成模型普遍面临空间推理困境，难以从提示中准确捕捉细粒度的空间关系，并正确生成具有结构完整性的场景。为缓解这一困境，我们提出了RL-RIG，一种基于反思的强化学习图像生成框架。我们的架构包含四个主要组件：扩散器、检查器、执行器和逆扩散器，遵循“生成-反思-编辑”范式，以激发图像生成中的思维链推理能力来解决该困境。为使模型对生成轨迹具备更好的直觉，我们进一步开发了Reflection-GRPO，分别用于训练VLM执行器生成编辑提示，以及训练图像编辑器在给定提示下提升图像质量。与仅生成视觉惊艳但结构不合理内容的传统方法不同，我们的评估指标优先考虑空间准确性，利用场景图IoU并采用VLM-as-a-Judge策略，在LAION-SG数据集上评估生成图像的空间一致性。实验结果表明，在图像生成的可控且精确的空间推理方面，RL-RIG比现有最先进的开源模型性能提升高达11%。