Although diffusion models can generate remarkably high-quality samples, they are intrinsically bottlenecked by their expensive iterative sampling procedure. Consistency models (CMs) have recently emerged as a promising diffusion model distillation method, reducing the cost of sampling by generating high-fidelity samples in just a few iterations. Consistency model distillation aims to solve the probability flow ordinary differential equation (ODE) defined by an existing diffusion model. CMs are not directly trained to minimize error against an ODE solver, rather they use a more computationally tractable objective. As a way to study how effectively CMs solve the probability flow ODE, and the effect that any induced error has on the quality of generated samples, we introduce Direct CMs, which \textit{directly} minimize this error. Intriguingly, we find that Direct CMs reduce the ODE solving error compared to CMs but also result in significantly worse sample quality, calling into question why exactly CMs work well in the first place. Full code is available at: https://github.com/layer6ai-labs/direct-cms.

翻译：尽管扩散模型能够生成质量极高的样本，但其本质上受到昂贵的迭代采样过程的限制。一致性模型（CMs）作为一种有前景的扩散模型蒸馏方法，通过仅需少数迭代即可生成高保真样本，从而降低了采样成本。一致性模型蒸馏旨在求解由现有扩散模型定义的概率流常微分方程（ODE）。CMs并非直接训练以最小化相对于ODE求解器的误差，而是采用计算上更易处理的目标函数。为了研究CMs求解概率流ODE的有效性，以及任何引入的误差对生成样本质量的影响，我们提出了直接一致性模型，该模型直接最小化此类误差。有趣的是，我们发现直接一致性模型相较于CMs降低了ODE求解误差，但同时也导致了显著更差的样本质量，这引发了对CMs最初为何能有效工作的根本性疑问。完整代码发布于：https://github.com/layer6ai-labs/direct-cms。