Diffusion-based text-to-image generation models have demonstrated strong performance in terms of image quality and diversity. However, they still struggle to generate images that accurately reflect the number of objects specified in the input prompt. Several approaches have been proposed that rely on either external counting modules for iterative refinement or quantity representations derived from learned tokens or latent features. However, they still have limitations in accurately reflecting the specified number of objects and overlook an important structural characteristic--The number of object instances in the generated image is largely determined in the early timesteps of the denoising process. To correctly reflect the object quantity for image generation, the highly activated regions in the object cross-attention map at the early timesteps should match the input object quantity, while each region should be clearly separated. To address this issue, we propose \textit{CountCluster}, a method that guides the object cross-attention map to be clustered according to the specified object count in the input, without relying on any external tools or additional training. The proposed method partitions the object cross-attention map into $k$ clusters at inference time based on attention scores, defines an ideal distribution in which each cluster is spatially well-separated, and optimizes the latent to align with this target distribution. Our method achieves an average improvement of 18.5\%p in object count accuracy compared to existing methods, and demonstrates superior quantity control performance across a variety of prompts. Code will be released at: https://github.com/JoohyeonL22/CountCluster

翻译：基于扩散的文本到图像生成模型在图像质量和多样性方面展现出强大性能。然而，这些模型在生成准确反映输入提示中指定对象数量的图像时仍面临困难。现有方法主要依赖两种策略：使用外部计数模块进行迭代优化，或基于学习到的标记或潜在特征构建数量表示。但这些方法在准确反映指定对象数量方面仍存在局限，且忽略了一个关键结构特征——生成图像中的对象实例数量主要取决于去噪过程的早期时间步。为实现图像生成中对象数量的准确控制，早期时间步的对象交叉注意力图中高度激活的区域应当与输入对象数量相匹配，且各区域之间需保持清晰分离。针对这一问题，本文提出 \textit{CountCluster} 方法，该方法无需依赖外部工具或额外训练，即可引导对象交叉注意力图根据输入指定的对象数量进行聚类。所提方法在推理阶段基于注意力分数将对象交叉注意力图划分为 $k$ 个聚类，定义每个聚类在空间上充分分离的理想分布，并通过优化潜在变量使其与目标分布对齐。实验表明，本方法在对象数量准确率上相比现有方法平均提升 18.5\%p，并在多样化提示场景中展现出卓越的数量控制性能。代码发布于：https://github.com/JoohyeonL22/CountCluster