Diffusion models have recently achieved remarkable progress in generating realistic images. However, challenges remain in accurately understanding and synthesizing the layout requirements in the textual prompts. To align the generated image with layout instructions, we present a training-free layout calibration system SimM that intervenes in the generative process on the fly during inference time. Specifically, following a "check-locate-rectify" pipeline, the system first analyses the prompt to generate the target layout and compares it with the intermediate outputs to automatically detect errors. Then, by moving the located activations and making intra- and inter-map adjustments, the rectification process can be performed with negligible computational overhead. To evaluate SimM over a range of layout requirements, we present a benchmark SimMBench that compensates for the lack of superlative spatial relations in existing datasets. And both quantitative and qualitative results demonstrate the effectiveness of the proposed SimM in calibrating the layout inconsistencies. Our project page is at https://simm-t2i.github.io/SimM.

翻译：扩散模型近期在生成逼真图像方面取得了显著进展。然而，在准确理解并合成文本提示中的布局要求方面仍存在挑战。为使生成图像与布局指令对齐，我们提出了一种无需训练的布局校准系统SimM，它在推理过程中即时干预生成过程。具体而言，遵循“检查-定位-修正”流程，该系统首先分析提示以生成目标布局，并将其与中间输出进行比较以自动检测错误。随后，通过移动所定位的激活并进行图内与图间调整，可以以可忽略的计算开销完成修正过程。为在多种布局要求下评估SimM，我们提出了一个基准SimMBench，弥补了现有数据集中缺乏强空间关系的不足。定量与定性结果均证明了SimM在校正布局不一致性方面的有效性。我们的项目页面位于https://simm-t2i.github.io/SimM。