Inverse problems arise in a multitude of applications, where the goal is to recover a clean signal from noisy and possibly (non)linear observations. The difficulty of a reconstruction problem depends on multiple factors, such as the structure of the ground truth signal, the severity of the degradation, the implicit bias of the reconstruction model and the complex interactions between the above factors. This results in natural sample-by-sample variation in the difficulty of a reconstruction task, which is often overlooked by contemporary techniques. Recently, diffusion-based inverse problem solvers have established new state-of-the-art in various reconstruction tasks. However, they have the drawback of being computationally prohibitive. Our key observation in this paper is that most existing solvers lack the ability to adapt their compute power to the difficulty of the reconstruction task, resulting in long inference times, subpar performance and wasteful resource allocation. We propose a novel method that we call severity encoding, to estimate the degradation severity of noisy, degraded signals in the latent space of an autoencoder. We show that the estimated severity has strong correlation with the true corruption level and can give useful hints at the difficulty of reconstruction problems on a sample-by-sample basis. Furthermore, we propose a reconstruction method based on latent diffusion models that leverages the predicted degradation severities to fine-tune the reverse diffusion sampling trajectory and thus achieve sample-adaptive inference times. We utilize latent diffusion posterior sampling to maintain data consistency with observations. We perform experiments on both linear and nonlinear inverse problems and demonstrate that our technique achieves performance comparable to state-of-the-art diffusion-based techniques, with significant improvements in computational efficiency.

翻译：逆问题广泛存在于多种应用场景中，其目标是从含噪且可能（非）线性的观测中恢复干净信号。重建问题的难度受多重因素影响，例如真实信号的结构、退化程度、重建模型的内隐偏差以及上述因素间的复杂相互作用。这导致重建任务在逐样本维度上自然存在难度差异，而现有技术往往忽视这一特性。近年来，基于扩散模型的逆问题求解器已在各类重建任务中创下新纪录，但其计算成本过高成为主要缺陷。本文的核心发现是：现有求解器大多缺乏根据任务难度动态调整计算资源的能力，导致推理时间过长、性能欠佳与资源浪费。我们提出一种名为“退化程度编码”的新方法，通过自编码器的潜在空间估计含噪退化信号的退化严重程度。实验表明，该估计值与真实退化水平高度相关，并能逐样本提供重建任务难度的有效提示。进一步，我们提出基于潜在扩散模型的重建方法，利用预测的退化程度精细调整反向扩散采样轨迹，从而实现样本自适应的推理时间。采用潜在扩散后验采样以保持观测数据一致性。在线性与非线性逆问题上的实验表明，本方法在计算效率显著提升的同时，性能可与先进的扩散方法媲美。