Real-World Image Super-Resolution is one of the most challenging task in image restoration. However, existing methods struggle with an accurate understanding of degraded image content, leading to reconstructed results that are both low-fidelity and unnatural. We present RealSR-R1 in this work, which empowers the RealSR models with understanding and reasoning capabilities. Inspired by the success of Chain of Thought (CoT) in large language models (LLMs), we simulate the human process of handling degraded images and propose the VLCoT framework, which integrates vision and language reasoning. The framework aims to precisely restore image details by progressively generating more comprehensive text and higher-resolution images. To overcome the challenge of traditional supervised learning CoT failing to generalize to real-world scenarios, we introduce, for the first time, Group Relative Policy Optimization (GRPO) into the Real-World Image Super-Resolution task. We propose VLCoT-GRPO as a solution, which designs four reward functions: (1) Format reward, used to standardize the CoT process; (2) Degradation reward, to incentivize accurate degradation estimation; (3) Understanding reward, to ensure the accuracy of the generated content; and (4) Generation reward, where we propose using a visual expert model to evaluate the quality of generated images, encouraging the model to generate more realistic images. Extensive experiments demonstrate that our proposed RealSR-R1 can generate realistic details and accurately understand image content, particularly in semantically rich scenes or images with severe degradation.
翻译:真实世界图像超分辨率是图像复原领域最具挑战性的任务之一。然而,现有方法难以准确理解退化图像的内容,导致重建结果既保真度低又不够自然。本文提出RealSR-R1,旨在赋予真实世界超分辨率模型理解与推理能力。受大语言模型中思维链(CoT)成功经验的启发,我们模拟人类处理退化图像的过程,提出了融合视觉与语言推理的VLCoT框架。该框架通过逐步生成更全面的文本描述和更高分辨率的图像,以实现对图像细节的精准复原。为克服传统监督学习思维链方法难以泛化至真实场景的挑战,我们首次将分组相对策略优化(GRPO)引入真实世界图像超分辨率任务,并提出VLCoT-GRPO解决方案。该方法设计了四项奖励函数:(1)格式奖励,用于规范思维链流程;(2)退化奖励,激励模型准确估计退化类型;(3)理解奖励,确保生成内容的准确性;(4)生成奖励,创新性地引入视觉专家模型评估生成图像质量,促使模型生成更逼真的图像。大量实验表明,我们提出的RealSR-R1能够生成逼真的细节并准确理解图像内容,尤其在语义丰富的场景或严重退化的图像中表现突出。