Image quality assessment (IQA) is very important for both end-users and service providers since a high-quality image can significantly improve the user's quality of experience (QoE) and also benefit lots of computer vision algorithms. Most existing blind image quality assessment (BIQA) models were developed for synthetically distorted images, however, they perform poorly on in-the-wild images, which are widely existed in various practical applications. In this paper, we propose a novel BIQA model for in-the-wild images by addressing two critical problems in this field: how to learn better quality-aware feature representation, and how to solve the problem of insufficient training samples in terms of their content and distortion diversity. Considering that perceptual visual quality is affected by both low-level visual features (e.g. distortions) and high-level semantic information (e.g. content), we first propose a staircase structure to hierarchically integrate the features from intermediate layers into the final feature representation, which enables the model to make full use of visual information from low-level to high-level. Then an iterative mixed database training (IMDT) strategy is proposed to train the BIQA model on multiple databases simultaneously, so the model can benefit from the increase in both training samples and image content and distortion diversity and can learn a more general feature representation. Experimental results show that the proposed model outperforms other state-of-the-art BIQA models on six in-the-wild IQA databases by a large margin. Moreover, the proposed model shows an excellent performance in the cross-database evaluation experiments, which further demonstrates that the learned feature representation is robust to images with diverse distortions and content. The code is available at https://github.com/sunwei925/StairIQA.
翻译:图像质量评估对终端用户和服务提供商均至关重要,因为高质量图像能显著提升用户体验质量,并惠及众多计算机视觉算法。现有大多数无参考图像质量评估模型主要针对合成失真图像设计,然而在广泛存在于各类实际应用场景的野外图像上表现欠佳。本文通过解决该领域的两个关键问题——如何学习更优的质量感知特征表示,以及如何解决训练样本在内容与失真多样性方面的不足——提出了一种新颖的野外图像无参考质量评估模型。考虑到感知视觉质量同时受低级视觉特征(如失真)和高级语义信息(如内容)影响,我们首先提出阶梯式结构,将中间层特征分层整合至最终特征表示,使模型能够充分利用从低级到高级的视觉信息。随后提出迭代混合数据库训练策略,在多个数据库上同时训练模型,使其受益于训练样本数量及图像内容与失真多样性的双重提升,从而学习到更具泛化能力的特征表示。实验结果表明,该模型在六个野外图像质量评估数据库上以显著优势优于现有主流无参考质量评估模型。此外,跨数据库评估实验显示该模型表现出色,进一步证明其学到的特征表示对多样化失真和内容图像具有鲁棒性。代码已开源:https://github.com/sunwei925/StairIQA。