Diffusion models have demonstrated empirical successes in various applications and can be adapted to task-specific needs via guidance. This paper studies a form of gradient guidance for adapting a pre-trained diffusion model towards optimizing user-specified objectives. We establish a mathematical framework for guided diffusion to systematically study its optimization theory and algorithmic design. Our theoretical analysis spots a strong link between guided diffusion models and optimization: gradient-guided diffusion models are essentially sampling solutions to a regularized optimization problem, where the regularization is imposed by the pre-training data. As for guidance design, directly bringing in the gradient of an external objective function as guidance would jeopardize the structure in generated samples. We investigate a modified form of gradient guidance based on a forward prediction loss, which leverages the information in pre-trained score functions and provably preserves the latent structure. We further consider an iteratively fine-tuned version of gradient-guided diffusion where guidance and score network are both updated with newly generated samples. This process mimics a first-order optimization iteration in expectation, for which we proved O(1/K) convergence rate to the global optimum when the objective function is concave. Our code will be released at https://github.com/yukang123/GGDMOptim.git.

翻译：扩散模型已在多种应用中展现出实证成功，并可通过引导机制适应任务特定需求。本文研究一种梯度引导形式，旨在将预训练扩散模型适配于用户指定目标的优化过程。我们建立了引导扩散的数学框架，以系统研究其优化理论与算法设计。理论分析揭示了引导扩散模型与优化之间的深刻联系：梯度引导扩散模型本质上是在对正则化优化问题的解进行采样，其中正则化项由预训练数据所施加。在引导设计方面，直接引入外部目标函数的梯度作为引导会破坏生成样本的结构特征。我们研究了一种基于前向预测损失的改进型梯度引导方法，该方法利用预训练评分函数中的信息，并在理论上保证潜在结构的保持。进一步提出梯度引导扩散的迭代微调版本，其中引导函数与评分网络均依据新生成样本进行更新。该过程在期望意义上模拟了一阶优化迭代，我们证明了当目标函数为凹函数时，该算法能以O(1/K)收敛速率达到全局最优解。代码发布于https://github.com/yukang123/GGDMOptim.git。