Learning-based image quality assessment (IQA) has made remarkable progress in the past decade, but nearly all consider the two key components -- model and data -- in isolation. Specifically, model-centric IQA focuses on developing ``better'' objective quality methods on fixed and extensively reused datasets, with a great danger of overfitting. Data-centric IQA involves conducting psychophysical experiments to construct ``better'' human-annotated datasets, which unfortunately ignores current IQA models during dataset creation. In this paper, we first design a series of experiments to probe computationally that such isolation of model and data impedes further progress of IQA. We then describe a computational framework that integrates model-centric and data-centric IQA. As a specific example, we design computational modules to quantify the sampling-worthiness of candidate images. Experimental results show that the proposed sampling-worthiness module successfully spots diverse failures of the examined blind IQA models, which are indeed worthy samples to be included in next-generation datasets.

翻译：基于学习的图像质量评估（IQA）在过去十年取得了显著进展，但几乎所有研究都将模型与数据这两个关键要素割裂开来。具体而言，模型中心IQA专注于在固定且被广泛重复使用的数据集上开发“更优”的客观质量方法，这极易导致过拟合问题。数据中心IQA则通过开展心理物理实验构建“更好”的人工标注数据集，但遗憾的是在数据集创建过程中忽略了当前的IQA模型。本文首先设计系列实验通过计算手段证明：模型与数据的割裂阻碍了IQA的进一步发展。随后我们提出一个融合模型中心与数据中心IQA的计算框架。作为具体实例，我们设计了量化候选图像采样价值的计算模块。实验结果表明，所提出的采样价值模块能够成功定位被检测的盲IQA模型中的各类失效情况，这些样本确实应当被纳入下一代数据集。