Diffusion models have recently revolutionized the field of image synthesis due to their ability to generate photorealistic images. However, one of the major drawbacks of diffusion models is that the image generation process is costly. A large image-to-image network has to be applied many times to iteratively refine an image from random noise. While many recent works propose techniques to reduce the number of required steps, they generally treat the underlying denoising network as a black box. In this work, we investigate the behavior of the layers within the network and find that 1) the layers' output changes smoothly over time, 2) the layers show distinct patterns of change, and 3) the change from step to step is often very small. We hypothesize that many layer computations in the denoising network are redundant. Leveraging this, we introduce block caching, in which we reuse outputs from layer blocks of previous steps to speed up inference. Furthermore, we propose a technique to automatically determine caching schedules based on each block's changes over timesteps. In our experiments, we show through FID, human evaluation and qualitative analysis that Block Caching allows to generate images with higher visual quality at the same computational cost. We demonstrate this for different state-of-the-art models (LDM and EMU) and solvers (DDIM and DPM).

翻译：扩散模型近期因能生成逼真图像而革新了图像合成领域。然而，其主要缺陷之一是图像生成过程计算成本高昂：需要多次应用大型图像到图像网络，从随机噪声迭代优化图像。尽管近期许多研究提出减少所需步骤的技术，但通常将底层去噪网络视为黑盒。本研究深入分析网络内各层行为，发现：1）各层输出随时间平滑变化；2）不同层呈现独特的变化模式；3）步间变化通常极小。我们假设去噪网络中的许多层计算存在冗余。基于此，我们提出块缓存方法，通过复用先前步骤中层块的输出来加速推理。此外，我们提出一种根据各块随时间变化自动确定缓存调度策略的技术。实验通过FID、人类评估和定性分析证明，在同等计算成本下，块缓存能生成视觉质量更高的图像。我们针对不同先进模型（LDM和EMU）及求解器（DDIM和DPM）验证了该方法的有效性。