Diffusion models have demonstrated empirical successes in various applications and can be adapted to task-specific needs via guidance. This paper introduces a form of gradient guidance for adapting or fine-tuning diffusion models towards user-specified optimization objectives. We study the theoretic aspects of a guided score-based sampling process, linking the gradient-guided diffusion model to first-order optimization. We show that adding gradient guidance to the sampling process of a pre-trained diffusion model is essentially equivalent to solving a regularized optimization problem, where the regularization term acts as a prior determined by the pre-training data. Diffusion models are able to learn data's latent subspace, however, explicitly adding the gradient of an external objective function to the sample process would jeopardize the structure in generated samples. To remedy this issue, we consider a modified form of gradient guidance based on a forward prediction loss, which leverages the pre-trained score function to preserve the latent structure in generated samples. We further consider an iteratively fine-tuned version of gradient-guided diffusion where one can query gradients at newly generated data points and update the score network using new samples. This process mimics a first-order optimization iteration in expectation, for which we proved O(1/K) convergence rate to the global optimum when the objective function is concave.

翻译：扩散模型已在多种应用中展现出实证成功，并可通过引导机制适配特定任务需求。本文引入一种梯度引导形式，用于将扩散模型适配或微调至用户指定的优化目标。我们从理论角度研究引导式基于分数的采样过程，将梯度引导扩散模型与一阶优化相关联。研究表明，向预训练扩散模型的采样过程添加梯度引导，本质上等价于求解一个正则化优化问题，其中正则化项充当由预训练数据决定的先验。扩散模型能够学习数据的潜在子空间，然而，在采样过程中显式添加外部目标函数的梯度会破坏生成样本中的结构。为解决此问题，我们考虑基于前向预测损失的梯度引导修正形式，该形式利用预训练分数函数保留生成样本中的潜在结构。进一步，我们提出迭代微调的梯度引导扩散版本，可查询新生成数据点的梯度，并利用新样本更新分数网络。该过程在期望意义上模拟一阶优化迭代，我们证明当目标函数为凹函数时，其收敛到全局最优的收敛率为O(1/K)。