Recently, diffusion model-based inverse problem solvers (DIS) have emerged as state-of-the-art approaches for addressing inverse problems, including image super-resolution, deblurring, inpainting, etc. However, their application to video inverse problems arising from spatio-temporal degradation remains largely unexplored due to the challenges in training video diffusion models. To address this issue, here we introduce an innovative video inverse solver that leverages only image diffusion models. Specifically, by drawing inspiration from the success of the recent decomposed diffusion sampler (DDS), our method treats the time dimension of a video as the batch dimension of image diffusion models and solves spatio-temporal optimization problems within denoised spatio-temporal batches derived from each image diffusion model. Moreover, we introduce a batch-consistent diffusion sampling strategy that encourages consistency across batches by synchronizing the stochastic noise components in image diffusion models. Our approach synergistically combines batch-consistent sampling with simultaneous optimization of denoised spatio-temporal batches at each reverse diffusion step, resulting in a novel and efficient diffusion sampling strategy for video inverse problems. Experimental results demonstrate that our method effectively addresses various spatio-temporal degradations in video inverse problems, achieving state-of-the-art reconstructions. Project page: https://svi-diffusion.github.io

翻译：近年来，基于扩散模型的逆问题求解器（DIS）已成为解决图像超分辨率、去模糊、修复等逆问题的最先进方法。然而，由于视频扩散模型训练面临挑战，其在时空退化引起的视频逆问题中的应用仍鲜有探索。为解决这一问题，本文提出一种仅利用图像扩散模型的创新视频逆求解器。具体而言，受近期分解扩散采样器（DDS）成功的启发，我们的方法将视频的时间维度视为图像扩散模型的批次维度，并在每个图像扩散模型导出的去噪时空批次内求解时空优化问题。此外，我们引入一种批次一致性扩散采样策略，通过同步图像扩散模型中的随机噪声分量来增强批次间的一致性。我们的方法将批次一致性采样与每个反向扩散步骤中时空去噪批次的同步优化协同结合，从而形成一种新颖高效的视频逆问题扩散采样策略。实验结果表明，我们的方法能有效处理视频逆问题中的各类时空退化，实现最先进的重建效果。项目页面：https://svi-diffusion.github.io