This paper introduces the first two pixel retrieval benchmarks. Pixel retrieval is segmented instance retrieval. Like semantic segmentation extends classification to the pixel level, pixel retrieval is an extension of image retrieval and offers information about which pixels are related to the query object. In addition to retrieving images for the given query, it helps users quickly identify the query object in true positive images and exclude false positive images by denoting the correlated pixels. Our user study results show pixel-level annotation can significantly improve the user experience. Compared with semantic and instance segmentation, pixel retrieval requires a fine-grained recognition capability for variable-granularity targets. To this end, we propose pixel retrieval benchmarks named PROxford and PRParis, which are based on the widely used image retrieval datasets, ROxford and RParis. Three professional annotators label 5,942 images with two rounds of double-checking and refinement. Furthermore, we conduct extensive experiments and analysis on the SOTA methods in image search, image matching, detection, segmentation, and dense matching using our pixel retrieval benchmarks. Results show that the pixel retrieval task is challenging to these approaches and distinctive from existing problems, suggesting that further research can advance the content-based pixel-retrieval and thus user search experience. The datasets can be downloaded from \href{https://github.com/anguoyuan/Pixel_retrieval-Segmented_instance_retrieval}{this link}.

翻译：本文首次提出了两个像素检索基准数据集。像素检索即分段实例检索。如同语义分割将分类任务扩展到像素级别，像素检索是图像检索的扩展，它能提供与查询对象相关的像素信息。除了检索与给定查询相关的图像外，像素检索通过标注相关像素，帮助用户快速识别真阳性图像中的查询对象，并排除假阳性图像。我们的用户研究结果表明，像素级标注能显著提升用户体验。与语义分割和实例分割相比，像素检索需要对可变粒度目标具有细粒度识别能力。为此，我们基于广泛使用的图像检索数据集ROxford和RParis提出了名为PROxford和PRParis的像素检索基准数据集。三位专业标注员历经两轮双重校验和精炼，对5,942张图像进行了标注。此外，我们利用像素检索基准数据集，对图像搜索、图像匹配、检测、分割及密集匹配领域的最先进方法进行了广泛的实验与分析。结果表明，像素检索任务对这些方法具有挑战性，且与现有问题存在显著差异，这表明进一步研究可推动基于内容的像素检索发展，进而改善用户搜索体验。该数据集可从以下链接下载：\href{https://github.com/anguoyuan/Pixel_retrieval-Segmented_instance_retrieval}{此链接}。