Diffusion models significantly improve the quality of super-resolved images with their impressive content generation capabilities. However, the huge computational costs limit the applications of these methods.Recent efforts have explored reasonable inference acceleration to reduce the number of sampling steps, but the computational cost remains high as each step is performed on the entire image.This paper introduces PatchScaler, a patch-independent diffusion-based single image super-resolution (SR) method, designed to enhance the efficiency of the inference process.The proposed method is motivated by the observation that not all the image patches within an image need the same sampling steps for reconstructing high-resolution images.Based on this observation, we thus develop a Patch-adaptive Group Sampling (PGS) to divide feature patches into different groups according to the patch-level reconstruction difficulty and dynamically assign an appropriate sampling configuration for each group so that the inference speed can be better accelerated.In addition, to improve the denoising ability at each step of the sampling, we develop a texture prompt to guide the estimations of the diffusion model by retrieving high-quality texture priors from a patch-independent reference texture memory.Experiments show that our PatchScaler achieves favorable performance in both quantitative and qualitative evaluations with fast inference speed.Our code and model are available at \url{https://github.com/yongliuy/PatchScaler}.

翻译：扩散模型凭借其卓越的内容生成能力，显著提升了超分辨率图像的质量。然而，巨大的计算成本限制了这些方法的应用。近期研究探索了合理的推理加速以减少采样步数，但由于每一步都在整张图像上执行，计算成本依然高昂。本文提出了PatchScaler，一种基于扩散的、与图像块无关的单图像超分辨率方法，旨在提升推理过程的效率。该方法的提出基于以下观察：图像中并非所有图像块都需要相同的采样步数来重建高分辨率图像。基于此观察，我们开发了一种块自适应分组采样方法，根据图像块级别的重建难度将特征块划分为不同组，并为每组动态分配合适的采样配置，从而更好地加速推理。此外，为了提升采样每一步的去噪能力，我们开发了一种纹理提示机制，通过从一个与图像块无关的参考纹理记忆中检索高质量纹理先验，来指导扩散模型的估计。实验表明，我们的PatchScaler在定量和定性评估中均取得了优越的性能，并具有快速的推理速度。我们的代码和模型可在 \url{https://github.com/yongliuy/PatchScaler} 获取。