Diffusion generative models unlock new possibilities for inverse problems as they allow for the incorporation of strong empirical priors into the process of scientific inference. Recently, diffusion models received significant attention for solving inverse problems by posterior sampling, but many challenges remain open due to the intractability of this sampling process. Prior work resorted to Gaussian approximations to conditional densities of the reverse process, leveraging Tweedie's formula to parameterise its mean, complemented with various heuristics. In this work, we leverage higher order information using Tweedie's formula and obtain a finer approximation with a principled covariance estimate. This novel approximation removes any time-dependent step-size hyperparameters required by earlier methods, and enables higher quality approximations of the posterior density which results in better samples. Specifically, we tackle noisy linear inverse problems and obtain a novel approximation to the gradient of the likelihood. We then plug this gradient estimate into various diffusion models and show that this method is optimal for a Gaussian data distribution. We illustrate the empirical effectiveness of our approach for general linear inverse problems on toy synthetic examples as well as image restoration using pretrained diffusion models as the prior. We show that our method improves the sample quality by providing statistically principled approximations to diffusion posterior sampling problem.

翻译：扩散生成模型为反问题开辟了新可能性，因其能将强大的经验先验融入科学推理过程。近年来，扩散模型通过后验采样解决反问题的方法受到广泛关注，但由于采样过程的难解性，许多挑战仍待解决。现有工作采用高斯近似处理逆向过程的条件密度，利用特威迪公式参数化其均值，并辅以多种启发式策略。本研究通过特威迪公式利用高阶信息，基于原则性的协方差估计获得更精细的近似。该新型近似消除了现有方法所需的时间步长相关超参数，同时实现后验密度的更高质量近似，从而生成更优样本。具体而言，我们针对噪声线性反问题，得到似然梯度的新型近似。将该梯度估计应用于各类扩散模型后，证明该方法对高斯数据分布具有最优性。通过玩具合成示例和基于预训练扩散先验的图像修复任务，我们验证了该方法在一般线性反问题中的实证有效性。实验表明，通过为扩散后验采样问题提供统计上原则性的近似，我们的方法显著提升了样本质量。