Diffusion models can learn strong image priors from underlying data distribution and use them to solve inverse problems, but the training process is computationally expensive and requires lots of data. Such bottlenecks prevent most existing works from being feasible for high-dimensional and high-resolution data such as 3D images. This paper proposes a method to learn an efficient data prior for the entire image by training diffusion models only on patches of images. Specifically, we propose a patch-based position-aware diffusion inverse solver, called PaDIS, where we obtain the score function of the whole image through scores of patches and their positional encoding and utilize this as the prior for solving inverse problems. First of all, we show that this diffusion model achieves an improved memory efficiency and data efficiency while still maintaining the capability to generate entire images via positional encoding. Additionally, the proposed PaDIS model is highly flexible and can be plugged in with different diffusion inverse solvers (DIS). We demonstrate that the proposed PaDIS approach enables solving various inverse problems in both natural and medical image domains, including CT reconstruction, deblurring, and superresolution, given only patch-based priors. Notably, PaDIS outperforms previous DIS methods trained on entire image priors in the case of limited training data, demonstrating the data efficiency of our proposed approach by learning patch-based prior.

翻译：扩散模型能够从底层数据分布中学习到强大的图像先验，并利用其解决逆问题，但训练过程计算成本高昂且需要大量数据。此类瓶颈使得现有大多数方法难以适用于高维和高分辨率数据（例如3D图像）。本文提出一种方法，仅通过在图像块上训练扩散模型，即可学习到针对整幅图像的高效数据先验。具体而言，我们提出一种基于块且具有位置感知的扩散逆求解器，称为PaDIS。在该方法中，我们通过图像块的分数及其位置编码来获得整幅图像的分数函数，并将其用作解决逆问题的先验。首先，我们证明该扩散模型在通过位置编码保持生成完整图像能力的同时，实现了更高的内存效率和数据效率。此外，所提出的PaDIS模型具有高度灵活性，可与不同的扩散逆求解器（DIS）结合使用。实验表明，在仅具备基于块的先验知识条件下，所提出的PaDIS方法能够解决自然图像和医学图像领域的多种逆问题，包括CT重建、去模糊和超分辨率。值得注意的是，在训练数据有限的情况下，PaDIS的性能优于以往基于完整图像先验训练的DIS方法，这证明了我们通过基于块的先验学习所实现的数据高效性。