Real-world Super-Resolution (real-SR) methods focus on dealing with diverse real-world images and have attracted increasing attention in recent years. The key idea is to use a complex and high-order degradation model to mimic real-world degradations. Although they have achieved impressive results in various scenarios, they are faced with the obstacle of evaluation. Currently, these methods are only assessed by their average performance on a small set of degradation cases randomly selected from a large space, which fails to provide a comprehensive understanding of their overall performance and often yields biased results. To overcome the limitation in evaluation, we propose SEAL, a framework for systematic evaluation of real-SR. In particular, we cluster the extensive degradation space to create a set of representative degradation cases, which serves as a comprehensive test set. Next, we propose a coarse-to-fine evaluation protocol to measure the distributed and relative performance of real-SR methods on the test set. The protocol incorporates two new metrics: acceptance rate (AR) and relative performance ratio (RPR), derived from an acceptance line and an excellence line. Under SEAL, we benchmark existing real-SR methods, obtain new observations and insights into their performance, and develop a new strong baseline. We consider SEAL as the first step towards creating an unbiased and comprehensive evaluation platform, which can promote the development of real-SR.

翻译：真实世界超分辨率（real-SR）方法专注于处理多样化的真实图像，近年来受到越来越多的关注。其核心思想是采用复杂高阶退化模型模拟真实场景中的退化过程。尽管这类方法已在多种场景中取得显著效果，却面临评估障碍。当前方法仅通过从大空间随机选取的小样本退化案例的平均性能进行评估，无法全面反映其整体表现，且常产生有偏结果。为克服评估局限性，我们提出SEAL框架——一个面向real-SR的系统性评估方案。具体而言，我们通过聚类构建广泛退化空间中的代表性退化案例集作为综合测试集，进而提出粗到细的评估协议，在测试集上量化real-SR方法的分布性能与相对性能。该协议引入两个新指标：接受率（AR）和相对性能比（RPR），分别基于接受线和卓越线推导得出。基于SEAL，我们对现有real-SR方法进行基准测试，获得关于其性能的新观察与见解，并开发出新的强基线方法。我们视SEAL为构建无偏且全面评估平台的第一步，以推动real-SR领域的发展。