Customizing diffusion models to generate identity-preserving images from user-provided reference images is an intriguing new problem. The prevalent approaches typically require training on extensive domain-specific images to achieve identity preservation, which lacks flexibility across different use cases. To address this issue, we exploit classifier guidance, a training-free technique that steers diffusion models using an existing classifier, for personalized image generation. Our study shows that based on a recent rectified flow framework, the major limitation of vanilla classifier guidance in requiring a special classifier can be resolved with a simple fixed-point solution, allowing flexible personalization with off-the-shelf image discriminators. Moreover, its solving procedure proves to be stable when anchored to a reference flow trajectory, with a convergence guarantee. The derived method is implemented on rectified flow with different off-the-shelf image discriminators, delivering advantageous personalization results for human faces, live subjects, and certain objects. Code is available at https://github.com/feifeiobama/RectifID.
翻译:定制扩散模型以根据用户提供的参考图像生成保持身份一致性的图像,是一个引人关注的新问题。现有方法通常需要在大量领域特定图像上进行训练才能实现身份保持,这在不同应用场景中缺乏灵活性。为解决此问题,我们探索了分类器引导这一无需训练的技术,该技术利用现有分类器来引导扩散模型,实现个性化图像生成。我们的研究表明,基于最新的修正流框架,原始分类器引导需要专用分类器的主要局限性可通过简单的定点求解方案解决,从而能够利用现成的图像判别器实现灵活个性化。此外,当锚定于参考流轨迹时,其求解过程被证明具有稳定性,并具备收敛性保证。所推导的方法在不同现成图像判别器上实现了修正流,为人脸、活体对象及特定物体提供了优越的个性化生成效果。代码发布于 https://github.com/feifeiobama/RectifID。