Denoising Diffusion Probabilistic Models (DDPMs) have garnered popularity for data generation across various domains. However, a significant bottleneck is the necessity for whole-network computation during every step of the generative process, leading to high computational overheads. This paper presents a novel framework, Denoising Diffusion Step-aware Models (DDSM), to address this challenge. Unlike conventional approaches, DDSM employs a spectrum of neural networks whose sizes are adapted according to the importance of each generative step, as determined through evolutionary search. This step-wise network variation effectively circumvents redundant computational efforts, particularly in less critical steps, thereby enhancing the efficiency of the diffusion model. Furthermore, the step-aware design can be seamlessly integrated with other efficiency-geared diffusion models such as DDIMs and latent diffusion, thus broadening the scope of computational savings. Empirical evaluations demonstrate that DDSM achieves computational savings of 49% for CIFAR-10, 61% for CelebA-HQ, 59% for LSUN-bedroom, 71% for AFHQ, and 76% for ImageNet, all without compromising the generation quality. Our code and models will be publicly available.

翻译：去噪扩散概率模型（DDPMs）因其在多个领域的数据生成能力而广受欢迎。然而，一个显著的瓶颈是生成过程的每一步都需要进行全网络计算，导致计算开销较高。本文提出了一种新颖的框架——去噪扩散步感知模型（DDSM）来应对这一挑战。与传统方法不同，DDSM采用一系列神经网络，其规模根据通过进化搜索确定的各生成步骤的重要性进行调整。这种步进式的网络变化有效规避了冗余计算，尤其是在不关键的步骤中，从而提升了扩散模型的效率。此外，步感知设计可与DDIMs和潜扩散等其他面向效率的扩散模型无缝集成，从而拓宽计算节省的范围。实验评估表明，DDSM在CIFAR-10上节省49%的计算量，在CelebA-HQ上节省61%，在LSUN-bedroom上节省59%，在AFHQ上节省71%，在ImageNet上节省76%，且均未影响生成质量。我们的代码和模型将公开提供。