Diffusion models have emerged as state-of-the-art generative models for image generation. However, sampling from diffusion models is usually time-consuming due to the inherent autoregressive nature of their sampling process. In this work, we propose a novel approach that accelerates the sampling of diffusion models by parallelizing the autoregressive process. Specifically, we reformulate the sampling process as solving a system of triangular nonlinear equations through fixed-point iteration. With this innovative formulation, we explore several systematic techniques to further reduce the iteration steps required by the solving process. Applying these techniques, we introduce ParaTAA, a universal and training-free parallel sampling algorithm that can leverage extra computational and memory resources to increase the sampling speed. Our experiments demonstrate that ParaTAA can decrease the inference steps required by common sequential sampling algorithms such as DDIM and DDPM by a factor of 4$\sim$14 times. Notably, when applying ParaTAA with 100 steps DDIM for Stable Diffusion, a widely-used text-to-image diffusion model, it can produce the same images as the sequential sampling in only 7 inference steps. The code is available at https://github.com/TZW1998/ParaTAA-Diffusion.

翻译：扩散模型已成为图像生成领域最先进的生成模型。然而，由于采样过程固有的自回归特性，从扩散模型中采样通常非常耗时。本文提出一种新颖方法，通过并行化自回归过程来加速扩散模型的采样。具体而言，我们将采样过程重新表述为通过定点迭代求解三角非线性方程组。基于这一创新性表述，我们探索了若干系统化技术以进一步减少求解过程所需的迭代步数。应用这些技术，我们提出了ParaTAA——一种通用且无需训练即可实现的并行采样算法，该算法能够利用额外的计算和内存资源来提升采样速度。实验表明，ParaTAA可将DDIM和DDPM等常见序列采样算法所需的推理步数减少4$\sim$14倍。值得注意的是，当将ParaTAA应用于广泛使用的文生图扩散模型Stable Diffusion，并配合100步DDIM采样时，仅需7个推理步即可生成与序列采样相同的图像。代码发布于https://github.com/TZW1998/ParaTAA-Diffusion。