To date, query performance prediction (QPP) in the context of content-based image retrieval remains a largely unexplored task, especially in the query-by-example scenario, where the query is an image. To boost the exploration of the QPP task in image retrieval, we propose the first benchmark for image query performance prediction (iQPP). First, we establish a set of four data sets (PASCAL VOC 2012, Caltech-101, ROxford5k and RParis6k) and estimate the ground-truth difficulty of each query as the average precision or the precision@k, using two state-of-the-art image retrieval models. Next, we propose and evaluate novel pre-retrieval and post-retrieval query performance predictors, comparing them with existing or adapted (from text to image) predictors. The empirical results show that most predictors do not generalize across evaluation scenarios. Our comprehensive experiments indicate that iQPP is a challenging benchmark, revealing an important research gap that needs to be addressed in future work. We release our code and data as open source at https://github.com/Eduard6421/iQPP, to foster future research.

翻译：迄今为止，在基于内容的图像检索中，查询性能预测（QPP）仍然是一个很大程度上未被探索的任务，尤其是在以图搜图的查询场景下。为推动图像检索中QPP任务的研究，我们提出了首个图像查询性能预测基准（iQPP）。首先，我们建立了四个数据集（PASCAL VOC 2012、Caltech-101、ROxford5k和RParis6k），并利用两种最先进的图像检索模型，以平均精度或精度@k估计每个查询的真实难度。接着，我们提出并评估了新颖的检索前与检索后查询性能预测器，将其与现有或经适配（从文本到图像）的预测器进行比较。实验结果表明，大多数预测器无法在不同评估场景中泛化。我们的综合实验显示，iQPP是一个具有挑战性的基准，揭示了未来研究中需要解决的重要研究空白。我们已在https://github.com/Eduard6421/iQPP 开源代码与数据，以促进后续研究。