To date, query performance prediction (QPP) in the context of content-based image retrieval remains a largely unexplored task, especially in the query-by-example scenario, where the query is an image. To boost the exploration of the QPP task in image retrieval, we propose the first benchmark for image query performance prediction (iQPP). First, we establish a set of four data sets (PASCAL VOC 2012, Caltech-101, ROxford5k and RParis6k) and estimate the ground-truth difficulty of each query as the average precision or the precision@k, using two state-of-the-art image retrieval models. Next, we propose and evaluate novel pre-retrieval and post-retrieval query performance predictors, comparing them with existing or adapted (from text to image) predictors. The empirical results show that most predictors do not generalize across evaluation scenarios. Our comprehensive experiments indicate that iQPP is a challenging benchmark, revealing an important research gap that needs to be addressed in future work. We release our code and data as open source at https://github.com/Eduard6421/iQPP, to foster future research.

翻译：迄今为止，基于内容的图像检索中的查询性能预测（QPP）仍是一个尚未充分探索的任务，特别是在以图搜图的查询示例场景中。为促进图像检索中QPP任务的研究，我们提出了首个图像查询性能预测基准（iQPP）。首先，我们建立了包含四个数据集（PASCAL VOC 2012、Caltech-101、ROxford5k和RParis6k）的基准集，并利用两种最先进的图像检索模型，以平均精度或top-k精度（precision@k）估计每个查询的真实难度。其次，我们提出并评估了新颖的检索前与检索后查询性能预测器，将其与现有预测器或从文本领域迁移至图像领域的预测器进行对比。实验结果表明，大多数预测器无法跨评估场景泛化。全面实验证明，iQPP是一个具有挑战性的基准，揭示了未来亟需填补的重要研究空白。我们已在https://github.com/Eduard6421/iQPP开源代码与数据，以推动后续研究。