Sampling from the posterior distribution poses a major computational challenge in solving inverse problems using latent diffusion models. Common methods rely on Tweedie's first-order moments, which are known to induce a quality-limiting bias. Existing second-order approximations are impractical due to prohibitive computational costs, making standard reverse diffusion processes intractable for posterior sampling. This paper introduces Second-order Tweedie sampler from Surrogate Loss (STSL), a novel sampler that offers efficiency comparable to first-order Tweedie with a tractable reverse process using second-order approximation. Our theoretical results reveal that the second-order approximation is lower bounded by our surrogate loss that only requires $O(1)$ compute using the trace of the Hessian, and by the lower bound we derive a new drift term to make the reverse process tractable. Our method surpasses SoTA solvers PSLD and P2L, achieving 4X and 8X reduction in neural function evaluations, respectively, while notably enhancing sampling quality on FFHQ, ImageNet, and COCO benchmarks. In addition, we show STSL extends to text-guided image editing and addresses residual distortions present from corrupted images in leading text-guided image editing methods. To our best knowledge, this is the first work to offer an efficient second-order approximation in solving inverse problems using latent diffusion and editing real-world images with corruptions.

翻译：从后验分布中采样是利用潜变量扩散模型求解逆问题时面临的主要计算挑战。现有方法普遍依赖Tweedie一阶矩，但此类方法存在导致采样质量下降的固有偏差。二阶近似方法虽已存在，却因计算成本过高而难以实用，这使得标准反向扩散过程无法用于后验采样。本文提出基于代理损失的二阶Tweedie采样器（STSL），该新型采样器采用可追踪的反向过程实现二阶近似，其计算效率可与一阶Tweedie方法媲美。理论分析表明，通过海森矩阵的迹仅需$O(1)$计算量即可获得二阶近似的下界，据此推导出的新型漂移项使反向过程变得可追踪。本方法在FFHQ、ImageNet和COCO基准测试中显著超越当前最优求解器PSLD和P2L，分别将神经网络评估次数降低4倍和8倍，同时显著提升采样质量。此外，我们证明STSL可扩展到文本引导图像编辑任务，有效消除主流文本引导图像编辑方法中由受损图像引起的残留失真。据我们所知，这是首个在基于潜变量扩散求解逆问题和处理真实图像污染中实现高效二阶近似的开创性工作。