While many diffusion models perform well when controlling particular aspects such as style, character, and interaction, they struggle with fine-grained control due to dataset limitations and intricate model architecture design. This paper introduces a novel training-free algorithm for fine-grained generation, called Aggregation of Multiple Diffusion Models (AMDM). The algorithm integrates features in the latent data space from multiple diffusion models within the same ecosystem into a specified model, thereby activating particular features and enabling fine-grained control. Experimental results demonstrate that AMDM significantly improves fine-grained control without training, validating its effectiveness. Additionally, it reveals that diffusion models initially focus on features such as position, attributes, and style, with later stages improving generation quality and consistency. AMDM offers a new perspective for tackling the challenges of fine-grained conditional generation in diffusion models. Specifically, it allows us to fully utilize existing or develop new conditional diffusion models that control specific aspects, and then aggregate them using the AMDM algorithm. This eliminates the need for constructing complex datasets, designing intricate model architectures, and incurring high training costs. Code is available at: https://github.com/Hammour-steak/AMDM.

翻译：尽管许多扩散模型在控制特定方面（如风格、角色和交互）表现良好，但由于数据集限制和复杂的模型架构设计，它们在细粒度控制方面仍存在困难。本文提出了一种新颖的无训练细粒度生成算法，称为多扩散模型聚合（AMDM）。该算法将同一生态系统中多个扩散模型的潜在数据空间特征整合到指定模型中，从而激活特定特征并实现细粒度控制。实验结果表明，AMDM在不进行训练的情况下显著提升了细粒度控制能力，验证了其有效性。此外，研究还发现扩散模型在初始阶段主要关注位置、属性和风格等特征，后期阶段则侧重于提升生成质量和一致性。AMDM为解决扩散模型中细粒度条件生成面临的挑战提供了新的视角。具体而言，它使我们能够充分利用现有或开发新的条件扩散模型来控制特定方面，然后通过AMDM算法进行聚合。这避免了构建复杂数据集、设计精细模型架构以及承担高昂训练成本的需求。代码可在以下网址获取：https://github.com/Hammour-steak/AMDM。