Customizing diffusion models to generate identity-preserving images from user-provided reference images is an intriguing new problem. The prevalent approaches typically require training on extensive domain-specific images to achieve identity preservation, which lacks flexibility across different use cases. To address this issue, we exploit classifier guidance, a training-free technique that steers diffusion models using an existing classifier, for personalized image generation. Our study shows that based on a recent rectified flow framework, the major limitation of vanilla classifier guidance in requiring a special classifier can be resolved with a simple fixed-point solution, allowing flexible personalization with off-the-shelf image discriminators. Moreover, its solving procedure proves to be stable when anchored to a reference flow trajectory, with a convergence guarantee. The derived method is implemented on rectified flow with different off-the-shelf image discriminators, delivering advantageous personalization results for human faces, live subjects, and certain objects. Code is available at https://github.com/feifeiobama/RectifID.
翻译:定制扩散模型以根据用户提供的参考图像生成保持身份特征的图像,是一个引人关注的新问题。现有方法通常需要在大量领域特定图像上进行训练才能实现身份保持,这在不同应用场景中缺乏灵活性。为解决这一问题,我们利用分类器引导这一无需训练的技术——通过现有分类器引导扩散模型——来实现个性化图像生成。我们的研究表明,基于最新的整流流框架,原始分类器引导需要特殊分类器的主要局限可通过简单的定点求解方案解决,从而能够利用现成的图像判别器实现灵活个性化。此外,当将其锚定在参考流轨迹上时,其求解过程被证明是稳定的,并具有收敛性保证。所推导的方法在整流流框架中结合多种现成图像判别器实现,在人物面部、活体对象及特定物体上均展现出优越的个性化生成效果。代码发布于 https://github.com/feifeiobama/RectifID。