Diffusion models, as a kind of powerful generative model, have given impressive results on image super-resolution (SR) tasks. However, due to the randomness introduced in the reverse process of diffusion models, the performances of diffusion-based SR models are fluctuating at every time of sampling, especially for samplers with few resampled steps. This inherent randomness of diffusion models results in ineffectiveness and instability, making it challenging for users to guarantee the quality of SR results. However, our work takes this randomness as an opportunity: fully analyzing and leveraging it leads to the construction of an effective plug-and-play sampling method that owns the potential to benefit a series of diffusion-based SR methods. More in detail, we propose to steadily sample high-quality SR images from pretrained diffusion-based SR models by solving diffusion ordinary differential equations (diffusion ODEs) with optimal boundary conditions (BCs) and analyze the characteristics between the choices of BCs and their corresponding SR results. Our analysis shows the route to obtain an approximately optimal BC via an efficient exploration in the whole space. The quality of SR results sampled by the proposed method with fewer steps outperforms the quality of results sampled by current methods with randomness from the same pretrained diffusion-based SR model, which means that our sampling method ``boosts'' current diffusion-based SR models without any additional training.
翻译:扩散模型作为一种强大的生成模型,已在图像超分辨率(SR)任务中取得显著成果。然而,由于扩散模型反向过程中引入的随机性,基于扩散的SR模型在每次采样时的性能波动较大,尤其在重采样步数较少的采样器中更为明显。这种固有随机性导致模型效率低下且不稳定,使用户难以保证SR结果的质量。然而,我们的工作将这种随机性视为机遇:通过充分分析并利用这种随机性,我们构建了一种有效的即插即用采样方法,该方法有望惠及一系列基于扩散的SR方法。具体而言,我们提出通过求解具有最优边界条件(BCs)的扩散常微分方程(扩散ODE),从预训练的基于扩散的SR模型中稳定采样高质量SR图像,并分析BCs选择与其对应SR结果之间的关系特征。我们的分析揭示了通过在整个空间中高效探索获得近似最优BC的路径。所提方法在更少步数下采样的SR结果质量,超过了从同一预训练扩散SR模型使用当前方法(含随机性)采样的结果质量,这意味着我们的采样方法无需额外训练即可“提升”现有基于扩散的SR模型。