Diffusion models have revolutionized the field of image generation, leading to the proliferation of high-quality models and diverse downstream applications. However, despite these significant advancements, the current competitive solutions still suffer from several limitations, including inferior visual quality, a lack of aesthetic appeal, and inefficient inference, without a comprehensive solution in sight. To address these challenges, we present UniFL, a unified framework that leverages feedback learning to enhance diffusion models comprehensively. UniFL stands out as a universal, effective, and generalizable solution applicable to various diffusion models, such as SD1.5 and SDXL. Notably, UniFL incorporates three key components: perceptual feedback learning, which enhances visual quality; decoupled feedback learning, which improves aesthetic appeal; and adversarial feedback learning, which optimizes inference speed. In-depth experiments and extensive user studies validate the superior performance of our proposed method in enhancing both the quality of generated models and their acceleration. For instance, UniFL surpasses ImageReward by 17% user preference in terms of generation quality and outperforms LCM and SDXL Turbo by 57% and 20% in 4-step inference. Moreover, we have verified the efficacy of our approach in downstream tasks, including Lora, ControlNet, and AnimateDiff.

翻译：扩散模型已经彻底改变了图像生成领域，推动了高质量模型的涌现及多样化的下游应用。然而，尽管取得了这些显著进展，当前的竞争性解决方案仍存在若干局限性，包括较差的视觉质量、缺乏美学吸引力以及推理效率低下，且尚无全面的解决方案。为应对这些挑战，我们提出UniFL，一个统一的框架，通过利用反馈学习全面增强扩散模型。UniFL作为一种通用、有效且可推广的解决方案，适用于多种扩散模型，如SD1.5和SDXL。值得注意的是，UniFL包含三个关键组件：感知反馈学习（提升视觉质量）、解耦反馈学习（改善美学吸引力）以及对抗反馈学习（优化推理速度）。深入的实验和广泛用户研究验证了我们所提方法在提升生成模型质量及其加速方面的优越性能。例如，UniFL在生成质量上超越ImageReward 17%的用户偏好，并在4步推理中分别超过LCM和SDXL Turbo 57%和20%。此外，我们验证了该方法在下游任务中的有效性，包括Lora、ControlNet和AnimateDiff。