Diffusion models have emerged as state-of-the-art generative models for image generation. However, sampling from diffusion models is usually time-consuming due to the inherent autoregressive nature of their sampling process. In this work, we propose a novel approach that accelerates the sampling of diffusion models by parallelizing the autoregressive process. Specifically, we reformulate the sampling process as solving a system of triangular nonlinear equations through fixed-point iteration. With this innovative formulation, we explore several systematic techniques to further reduce the iteration steps required by the solving process. Applying these techniques, we introduce ParaTAA, a universal and training-free parallel sampling algorithm that can leverage extra computational and memory resources to increase the sampling speed. Our experiments demonstrate that ParaTAA can decrease the inference steps required by common sequential sampling algorithms such as DDIM and DDPM by a factor of 4~14 times. Notably, when applying ParaTAA with 100 steps DDIM for Stable Diffusion, a widely-used text-to-image diffusion model, it can produce the same images as the sequential sampling in only 7 inference steps.

翻译：扩散模型已成为图像生成领域最先进的生成模型。然而，由于采样过程固有的自回归特性，从扩散模型中采样通常非常耗时。本文提出了一种新颖方法，通过并行化自回归过程来加速扩散模型的采样。具体来说，我们将采样过程重新表述为通过不动点迭代求解三角非线性方程组。基于这一创新性表述，我们探索了多种系统化技术以进一步减少求解过程所需的迭代步数。应用这些技术，我们提出了ParaTAA——一种通用且无需训练的并行采样算法，该算法能够利用额外的计算和内存资源来提高采样速度。实验表明，与DDIM和DDPM等常见顺序采样算法相比，ParaTAA可将推理步数减少4~14倍。值得注意的是，将ParaTAA与100步DDIM应用于广泛使用的文本到图像扩散模型Stable Diffusion时，仅需7个推理步即可生成与顺序采样相同的图像。