Recently, there have been significant improvements in the quality and performance of text-to-image generation, largely due to the impressive results attained by diffusion models. However, text-to-image diffusion models sometimes struggle to create high-fidelity content for the given input prompt. One specific issue is their difficulty in generating the precise number of objects specified in the text prompt. For example, when provided with the prompt "five apples and ten lemons on a table," images generated by diffusion models often contain an incorrect number of objects. In this paper, we present a method to improve diffusion models so that they accurately produce the correct object count based on the input prompt. We adopt a counting network that performs reference-less class-agnostic counting for any given image. We calculate the gradients of the counting network and refine the predicted noise for each step. To address the presence of multiple types of objects in the prompt, we utilize novel attention map guidance to obtain high-quality masks for each object. Finally, we guide the denoising process using the calculated gradients for each object. Through extensive experiments and evaluation, we demonstrate that the proposed method significantly enhances the fidelity of diffusion models with respect to object count.

翻译：近年来，文本到图像生成的质量和性能得到了显著提升，这主要得益于扩散模型所取得的令人瞩目的成果。然而，文本到图像扩散模型有时难以根据给定的输入提示生成高保真的内容。一个具体的问题是，它们在生成文本提示中指定的精确物体数量方面存在困难。例如，当输入提示为"桌子上的五个苹果和十个柠檬"时，扩散模型生成的图像通常包含错误数量的物体。在本文中，我们提出了一种改进扩散模型的方法，使其能够根据输入提示准确地生成正确数量的物体。我们采用了一个计数网络，该网络可以对任意给定图像执行无参考的、与类别无关的计数。我们计算该计数网络的梯度，并细化每一步的预测噪声。为了解决提示中存在多种类型物体的问题，我们利用新颖的注意力图引导技术来获取每个物体的高质量掩码。最后，我们使用计算出的每个物体的梯度来引导去噪过程。通过大量的实验和评估，我们证明了所提出的方法在物体数量方面显著增强了扩散模型的保真度。