Building on the success of diffusion models in visual generation, flow-based models reemerge as another prominent family of generative models that have achieved competitive or better performance in terms of both visual quality and inference speed. By learning the velocity field through flow-matching, flow-based models tend to produce a straighter sampling trajectory, which is advantageous during the sampling process. However, unlike diffusion models for which fast samplers are well-developed, efficient sampling of flow-based generative models has been rarely explored. In this paper, we propose a framework called FlowTurbo to accelerate the sampling of flow-based models while still enhancing the sampling quality. Our primary observation is that the velocity predictor's outputs in the flow-based models will become stable during the sampling, enabling the estimation of velocity via a lightweight velocity refiner. Additionally, we introduce several techniques including a pseudo corrector and sample-aware compilation to further reduce inference time. Since FlowTurbo does not change the multi-step sampling paradigm, it can be effectively applied for various tasks such as image editing, inpainting, etc. By integrating FlowTurbo into different flow-based models, we obtain an acceleration ratio of 53.1%$\sim$58.3% on class-conditional generation and 29.8%$\sim$38.5% on text-to-image generation. Notably, FlowTurbo reaches an FID of 2.12 on ImageNet with 100 (ms / img) and FID of 3.93 with 38 (ms / img), achieving the real-time image generation and establishing the new state-of-the-art. Code is available at https://github.com/shiml20/FlowTurbo.
翻译:基于扩散模型在视觉生成领域的成功,基于流的模型作为另一类重要的生成模型重新崛起,在视觉质量和推理速度方面均展现出竞争性乃至更优的性能。通过流匹配学习速度场,基于流的模型倾向于产生更直的采样轨迹,这在采样过程中具有优势。然而,与已发展出快速采样器的扩散模型不同,基于流的生成模型的高效采样方法鲜有探索。本文提出一个名为FlowTurbo的框架,旨在加速基于流模型的采样过程,同时提升采样质量。我们的主要观察是,基于流模型中速度预测器的输出在采样过程中会趋于稳定,从而使得通过轻量级速度精炼器估计速度成为可能。此外,我们引入了包括伪校正器和样本感知编译在内的多项技术,以进一步减少推理时间。由于FlowTurbo不改变多步采样范式,它可以有效地应用于图像编辑、修复等多种任务。通过将FlowTurbo集成到不同的基于流模型中,我们在类别条件生成任务上获得了53.1%$\sim$58.3%的加速比,在文本到图像生成任务上获得了29.8%$\sim$38.5%的加速比。值得注意的是,FlowTurbo在ImageNet上以100(毫秒/图像)的推理时间达到了2.12的FID分数,以38(毫秒/图像)达到了3.93的FID分数,实现了实时图像生成,并创造了新的最优性能。代码可在https://github.com/shiml20/FlowTurbo获取。