For an artist or a graphic designer, the spatial layout of a scene is a critical design choice. However, existing text-to-image diffusion models provide limited support for incorporating spatial information. This paper introduces Composite Diffusion as a means for artists to generate high-quality images by composing from the sub-scenes. The artists can specify the arrangement of these sub-scenes through a flexible free-form segment layout. They can describe the content of each sub-scene primarily using natural text and additionally by utilizing reference images or control inputs such as line art, scribbles, human pose, canny edges, and more. We provide a comprehensive and modular method for Composite Diffusion that enables alternative ways of generating, composing, and harmonizing sub-scenes. Further, we wish to evaluate the composite image for effectiveness in both image quality and achieving the artist's intent. We argue that existing image quality metrics lack a holistic evaluation of image composites. To address this, we propose novel quality criteria especially relevant to composite generation. We believe that our approach provides an intuitive method of art creation. Through extensive user surveys, quantitative and qualitative analysis, we show how it achieves greater spatial, semantic, and creative control over image generation. In addition, our methods do not need to retrain or modify the architecture of the base diffusion models and can work in a plug-and-play manner with the fine-tuned models.

翻译：对于艺术家或平面设计师而言，场景的空间布局是一项关键设计选择。然而，现有的文本到图像扩散模型在融入空间信息方面提供的支持有限。本文提出复合扩散方法，使艺术家能够通过组合子场景生成高质量图像。艺术家可通过灵活的自由形式片段布局指定这些子场景的排列方式，主要利用自然文本描述各子场景内容，并辅以参考图像或线稿、涂鸦、人体姿态、Canny边缘等控制输入。我们提供一种全面且模块化的复合扩散方法，支持子场景生成、组合与和谐化的多种替代方案。此外，我们希望从图像质量与艺术家意图达成度两方面评估复合图像的有效性。我们认为现有图像质量指标缺乏对图像复合的整体性评价。为此，我们提出特别针对复合生成的新型质量准则。我们相信该方法提供了直观的艺术创作方式。通过广泛用户调查、定量与定性分析，我们展示了该方法如何在图像生成中实现更强的空间、语义与创造性控制。此外，我们的方法无需重新训练或修改基础扩散模型的架构，并能以即插即用方式适配微调模型。