Diffusion generative models unlock new possibilities for inverse problems as they allow for the incorporation of strong empirical priors into the process of scientific inference. Recently, diffusion models received significant attention for solving inverse problems by posterior sampling, but many challenges remain open due to the intractability of this sampling process. Prior work resorted to Gaussian approximations to conditional densities of the reverse process, leveraging Tweedie's formula to parameterise its mean, complemented with various heuristics. In this work, we leverage higher order information using Tweedie's formula and obtain a finer approximation with a principled covariance estimate. This novel approximation removes any time-dependent step-size hyperparameters required by earlier methods, and enables higher quality approximations of the posterior density which results in better samples. Specifically, we tackle noisy linear inverse problems and obtain a novel approximation to the gradient of the likelihood. We then plug this gradient estimate into various diffusion models and show that this method is optimal for a Gaussian data distribution. We illustrate the empirical effectiveness of our approach for general linear inverse problems on toy synthetic examples as well as image restoration using pretrained diffusion models as the prior. We show that our method improves the sample quality by providing statistically principled approximations to diffusion posterior sampling problem.

翻译：扩散生成模型为逆问题带来了新的可能性，因为它们允许将强大的经验先验引入科学推断过程。近年来，扩散模型通过后验采样解决逆问题引起了广泛关注，但由于该采样过程的难解性，仍有许多开放性挑战。先前的工作采用高斯近似对逆过程的条件密度进行建模，利用Tweedie公式参数化其均值，并辅以各种启发式方法。本文利用Tweedie公式获取高阶信息，通过一个具有理论依据的协方差估计实现了更精细的近似。这种新颖的近似消除了早期方法所需的时间依赖步长超参数，并能够获得更高质量的后验密度近似，从而生成更优的样本。具体而言，我们针对含噪线性逆问题，提出了一种新的似然梯度近似方法。随后将该梯度估计应用于多种扩散模型，并证明该方法在高斯数据分布下具有最优性。我们通过玩具合成示例上的通用线性逆问题以及利用预训练扩散模型作为先验的图像复原任务，展示了该方法在经验上的有效性。结果表明，我们的方法通过为扩散后验采样问题提供统计上严谨的近似，显著提升了样本质量。