We aim at advancing blind image quality assessment (BIQA), which predicts the human perception of image quality without any reference information. We develop a general and automated multitask learning scheme for BIQA to exploit auxiliary knowledge from other tasks, in a way that the model parameter sharing and the loss weighting are determined automatically. Specifically, we first describe all candidate label combinations (from multiple tasks) using a textual template, and compute the joint probability from the cosine similarities of the visual-textual embeddings. Predictions of each task can be inferred from the joint distribution, and optimized by carefully designed loss functions. Through comprehensive experiments on learning three tasks - BIQA, scene classification, and distortion type identification, we verify that the proposed BIQA method 1) benefits from the scene classification and distortion type identification tasks and outperforms the state-of-the-art on multiple IQA datasets, 2) is more robust in the group maximum differentiation competition, and 3) realigns the quality annotations from different IQA datasets more effectively. The source code is available at https://github.com/zwx8981/LIQE.
翻译:我们旨在推进盲图像质量评估(BIQA),该任务无需任何参考信息即可预测人类对图像质量的感知。我们为BIQA开发了一种通用且自动化的多任务学习方案,以自动确定模型参数共享和损失权重的方式,利用其他任务的辅助知识。具体而言,我们首先使用文本模板描述(来自多个任务的)所有候选标签组合,并通过视觉-文本嵌入的余弦相似度计算联合概率。每个任务的预测可从联合分布中推断,并通过精心设计的损失函数进行优化。通过对三项任务(BIQA、场景分类和失真类型识别)的综合实验,我们验证了所提出的BIQA方法:1)得益于场景分类和失真类型识别任务,在多个IQA数据集上优于现有最先进方法;2)在群体最大差异化竞争中更具鲁棒性;3)更有效地重新对齐不同IQA数据集的质量标注。源代码已开源至https://github.com/zwx8981/LIQE。