Recent inverse problem solvers that leverage generative diffusion priors have garnered significant attention due to their exceptional quality. However, adaptation of the prior is necessary when there exists a discrepancy between the training and testing distributions. In this work, we propose deep diffusion image prior (DDIP), which generalizes the recent adaptation method of SCD by introducing a formal connection to the deep image prior. Under this framework, we propose an efficient adaptation method dubbed D3IP, specified for 3D measurements, which accelerates DDIP by orders of magnitude while achieving superior performance. D3IP enables seamless integration of 3D inverse solvers and thus leads to coherent 3D reconstruction. Moreover, we show that meta-learning techniques can also be applied to yield even better performance. We show that our method is capable of solving diverse 3D reconstructive tasks from the generative prior trained only with phantom images that are vastly different from the training set, opening up new opportunities of applying diffusion inverse solvers even when training with gold standard data is impossible. Code: https://github.com/HJ-harry/DDIP3D

翻译：近年来，利用生成扩散先验的逆问题求解器因其卓越的重建质量而受到广泛关注。然而，当训练分布与测试分布之间存在差异时，对先验模型进行适应调整是必要的。本文提出深度扩散图像先验（DDIP），该方法通过建立与深度图像先验的形式化联系，推广了近期SCD适应方法。在此框架下，我们提出了一种针对三维测量的高效适应方法D3IP，该方法将DDIP加速数个数量级的同时实现了更优的性能。D3IP实现了三维逆问题求解器的无缝集成，从而获得连贯的三维重建结果。此外，我们证明元学习技术亦可应用于此框架以取得更佳性能。实验表明，我们的方法能够利用仅在幻像图像上训练的生成先验，解决多样化的三维重建任务，这些幻像图像与训练集差异显著。这为扩散逆问题求解器开辟了新的应用场景，即使在无法获取黄金标准训练数据的情况下亦可部署。代码：https://github.com/HJ-harry/DDIP3D