Multi-modality foundation models, as represented by GPT-4V, have brought a new paradigm for low-level visual perception and understanding tasks, that can respond to a broad range of natural human instructions in a model. While existing foundation models have shown exciting potentials on low-level visual tasks, their related abilities are still preliminary and need to be improved. In order to enhance these models, we conduct a large-scale subjective experiment collecting a vast number of real human feedbacks on low-level vision. Each feedback follows a pathway that starts with a detailed description on the low-level visual appearance (*e.g. clarity, color, brightness* of an image, and ends with an overall conclusion, with an average length of 45 words. The constructed **Q-Pathway** dataset includes 58K detailed human feedbacks on 18,973 images with diverse low-level appearance. Moreover, to enable foundation models to robustly respond to diverse types of questions, we design a GPT-participated conversion to process these feedbacks into diverse-format 200K instruction-response pairs. Experimental results indicate that the **Q-Instruct** consistently elevates low-level perception and understanding abilities across several foundational models. We anticipate that our datasets can pave the way for a future that general intelligence can perceive, understand low-level visual appearance and evaluate visual quality like a human. Our dataset, model zoo, and demo is published at: https://q-future.github.io/Q-Instruct.

翻译：多模态基础模型（以GPT-4V为代表）为低级视觉感知与理解任务带来了全新范式，使单一模型能够响应人类多样化的自然语言指令。尽管现有基础模型在低级视觉任务中展现出令人兴奋的潜力，但其相关能力仍处于初级阶段，亟待提升。为增强这些模型，我们开展了一项大规模主观实验，收集了大量关于低级视觉的真实人类反馈。每条反馈遵循特定路径：首先对低级视觉外观（如图像的清晰度、色彩、亮度）进行详细描述，最终给出整体结论，平均长度为45词。构建的**Q-Pathway**数据集包含58K条针对18,973张具有多样低级外观图像的人类详细反馈。此外，为使基础模型能够稳健响应各类问题，我们设计了GPT参与的转换流程，将这些反馈处理为20万对多样化格式的指令-响应对。实验结果表明，**Q-Instruct**能持续提升多个基础模型的低级感知与理解能力。我们期待该数据集能为通用智能体像人类一样感知、理解低级视觉外观并评估视觉质量铺平道路。数据集、模型库及演示链接：https://q-future.github.io/Q-Instruct。