Diffusion models have demonstrated impressive performance in various image generation, editing, enhancement and translation tasks. In particular, the pre-trained text-to-image stable diffusion models provide a potential solution to the challenging realistic image super-resolution (Real-ISR) and image stylization problems with their strong generative priors. However, the existing methods along this line often fail to keep faithful pixel-wise image structures. If extra skip connections are used to reproduce details, additional training in image space will be required, limiting the application to tasks in latent space such as image stylization. In this work, we propose a pixel-aware stable diffusion (PASD) network to achieve robust Real-ISR and personalized image stylization. Specifically, a pixel-aware cross attention module is introduced to enable diffusion models perceiving image local structures in pixel-wise level, while a degradation removal module is used to extract degradation insensitive features to guide the diffusion process together with image high level information. An adjustable noise schedule is introduced to further improve the image restoration results. By simply replacing the base diffusion model with a stylized one, PASD can generate diverse stylized images without collecting pairwise training data, and by shifting the base model with an aesthetic one, PASD can bring old photos back to life. Extensive experiments in a variety of image enhancement and stylization tasks demonstrate the effectiveness of our proposed PASD approach. Our source codes are available at \url{https://github.com/yangxy/PASD/}.

翻译：扩散模型在图像生成、编辑、增强和翻译等多项任务中展现了卓越性能。特别是预训练的文本到图像稳定扩散模型，凭借其强大的生成先验，为解决极具挑战性的真实图像超分辨率（Real-ISR）和图像风格化问题提供了潜在方案。然而，现有方法在保持像素级图像结构保真度方面仍存在不足。若采用额外跳跃连接还原细节，则需要在图像空间中进行额外训练，从而限制了其在潜在空间任务（如图像风格化）中的应用。本文提出像素感知稳定扩散（PASD）网络，旨在实现鲁棒的真实图像超分辨率与个性化图像风格化。具体而言，我们引入像素感知交叉注意力模块，使扩散模型能够感知像素级的图像局部结构；同时设计退化移除模块，提取退化不敏感特征，结合图像高层信息引导扩散过程。进一步提出可调噪声调度策略以提升图像恢复效果。通过将基础扩散模型替换为风格化模型，PASD无需成对训练数据即可生成多样化的风格化图像；若替换为美学模型，则可使老照片焕发新生。在多种图像增强与风格化任务上的大量实验证实了所提PASD方法的有效性。源代码已公开于\url{https://github.com/yangxy/PASD/}。