Current diffusion-based super-resolution (SR) approaches achieve commendable performance at the cost of high inference overhead. Therefore, distillation techniques are utilized to accelerate the multi-step teacher model into one-step student model. Nevertheless, these methods significantly raise training costs and constrain the performance of the student model by the teacher model. To overcome these tough challenges, we propose Consistency Trajectory Matching for Super-Resolution (CTMSR), a distillation-free strategy that is able to generate photo-realistic SR results in one step. Concretely, we first formulate a Probability Flow Ordinary Differential Equation (PF-ODE) trajectory to establish a deterministic mapping from low-resolution (LR) images with noise to high-resolution (HR) images. Then we apply the Consistency Training (CT) strategy to directly learn the mapping in one step, eliminating the necessity of pre-trained diffusion model. To further enhance the performance and better leverage the ground-truth during the training process, we aim to align the distribution of SR results more closely with that of the natural images. To this end, we propose to minimize the discrepancy between their respective PF-ODE trajectories from the LR image distribution by our meticulously designed Distribution Trajectory Matching (DTM) loss, resulting in improved realism of our recovered HR images. Comprehensive experimental results demonstrate that the proposed methods can attain comparable or even superior capabilities on both synthetic and real datasets while maintaining minimal inference latency.
翻译:当前基于扩散模型的超分辨率方法以高推理开销为代价取得了令人瞩目的性能。因此,蒸馏技术被用于将多步教师模型加速为单步学生模型。然而,这些方法显著增加了训练成本,且学生模型的性能受限于教师模型。为克服这些严峻挑战,我们提出用于超分辨率的一致性轨迹匹配方法——一种无需蒸馏的策略,能够单步生成照片级真实感的超分辨率结果。具体而言,我们首先构建概率流常微分方程轨迹,以建立从含噪声的低分辨率图像到高分辨率图像的确定性映射。随后,我们应用一致性训练策略直接学习单步映射,从而无需预训练扩散模型。为进一步提升性能并更好地利用训练过程中的真实高分辨率图像,我们致力于使超分辨率结果的分布更贴近自然图像的分布。为此,我们提出通过精心设计的分布轨迹匹配损失来最小化二者从低分辨率图像分布出发的各自概率流常微分方程轨迹之间的差异,从而提升重建高分辨率图像的真实感。综合实验结果表明,所提方法在合成与真实数据集上均能获得相当甚至更优的性能,同时保持极低的推理延迟。