To date, query performance prediction (QPP) in the context of content-based image retrieval remains a largely unexplored task, especially in the query-by-example scenario, where the query is an image. To boost the exploration of the QPP task in image retrieval, we propose the first benchmark for image query performance prediction (iQPP). First, we establish a set of four data sets (PASCAL VOC 2012, Caltech-101, ROxford5k and RParis6k) and estimate the ground-truth difficulty of each query as the average precision or the precision@k, using two state-of-the-art image retrieval models. Next, we propose and evaluate novel pre-retrieval and post-retrieval query performance predictors, comparing them with existing or adapted (from text to image) predictors. The empirical results show that most predictors do not generalize across evaluation scenarios. Our comprehensive experiments indicate that iQPP is a challenging benchmark, revealing an important research gap that needs to be addressed in future work. We release our code and data as open source at https://github.com/Eduard6421/iQPP, to foster future research.

翻译：迄今为止，基于内容的图像检索中的查询性能预测（QPP）仍是一个尚未充分探索的任务，尤其是在以图像为查询的按例查询场景中。为促进图像检索中QPP任务的研究，我们提出了首个图像查询性能预测基准（iQPP）。首先，我们建立了四个数据集（PASCAL VOC 2012、Caltech-101、ROxford5k和RParis6k），并使用两个最先进的图像检索模型，以平均精度或precision@k估计每个查询的真实难度。接着，我们提出并评估了新颖的检索前和检索后查询性能预测器，并将其与现有或经改编（从文本到图像）的预测器进行比较。实证结果表明，大多数预测器无法跨评估场景泛化。我们的综合实验表明，iQPP是一个具有挑战性的基准，揭示了未来研究中需要解决的重要研究空白。我们已在https://github.com/Eduard6421/iQPP上开源代码和数据，以推动未来研究。