This paper presents a summary of the VQualA 2025 Challenge on Visual Quality Comparison for Large Multimodal Models (LMMs), hosted as part of the ICCV 2025 Workshop on Visual Quality Assessment. The challenge aims to evaluate and enhance the ability of state-of-the-art LMMs to perform open-ended and detailed reasoning about visual quality differences across multiple images. To this end, the competition introduces a novel benchmark comprising thousands of coarse-to-fine grained visual quality comparison tasks, spanning single images, pairs, and multi-image groups. Each task requires models to provide accurate quality judgments. The competition emphasizes holistic evaluation protocols, including 2AFC-based binary preference and multi-choice questions (MCQs). Around 100 participants submitted entries, with five models demonstrating the emerging capabilities of instruction-tuned LMMs on quality assessment. This challenge marks a significant step toward open-domain visual quality reasoning and comparison and serves as a catalyst for future research on interpretable and human-aligned quality evaluation systems.

翻译：本文概述了在ICCV 2025视觉质量评估研讨会期间举办的VQualA 2025挑战赛，该挑战赛聚焦于大型多模态模型的视觉质量比较。本挑战赛旨在评估并提升当前最先进的大型多模态模型在跨多张图像进行开放式、细粒度视觉质量差异推理的能力。为此，竞赛引入了一个新颖的基准测试，包含数千个从粗粒度到细粒度的视觉质量比较任务，涵盖单张图像、图像对以及多图像组。每项任务均要求模型提供准确的质量判断。竞赛强调全面的评估协议，包括基于二项迫选法的偏好判断和多项选择题。约有100名参赛者提交了方案，其中五个模型展示了指令微调后的大型多模态模型在质量评估方面的新兴能力。本次挑战赛标志着向开放域视觉质量推理与比较迈出了重要一步，并为未来可解释且与人类对齐的质量评估系统的研究起到了催化作用。