Query performance prediction (QPP) is an important and actively studied information retrieval task, having various applications, such as query reformulation, query expansion, and retrieval system selection, among many others. The task has been primarily studied in the context of text and image retrieval, whereas QPP for content-based video retrieval (CBVR) remains largely underexplored. To this end, we propose the first benchmark for video query performance prediction (VQPP), comprising two text-to-video retrieval datasets and two CBVR systems, respectively. VQPP contains a total of 56K text queries and 51K videos, and comes with official training, validation and test splits, fostering direct comparisons and reproducible results. We explore multiple pre-retrieval and post-retrieval performance predictors, creating a representative benchmark for future exploration of QPP in the video domain. Our results show that pre-retrieval predictors obtain competitive performance, enabling applications before performing the retrieval step. We also demonstrate the applicability of VQPP by employing the best performing pre-retrieval predictor as reward model for training a large language model (LLM) on the query reformulation task via direct preference optimization (DPO). We release our benchmark and code at https://github.com/AdrianLutu/VQPP.

翻译：查询性能预测（QPP）是信息检索领域中一项重要且被广泛研究的任务，具有多种应用，例如查询重构、查询扩展和检索系统选择等。该任务主要在文本和图像检索的背景下进行研究，而基于内容的视频检索（CBVR）中的QPP在很大程度上仍未得到充分探索。为此，我们提出了首个视频查询性能预测（VQPP）基准，分别包含两个文本到视频检索数据集和两个CBVR系统。VQPP总共包含56K个文本查询和51K个视频，并提供了官方的训练、验证和测试划分，以促进直接比较和可复现的结果。我们探索了多种检索前和检索后性能预测器，为未来在视频领域探索QPP创建了一个具有代表性的基准。我们的结果表明，检索前预测器能够获得具有竞争力的性能，从而使得在检索步骤执行之前即可进行相关应用。我们还通过将性能最佳的检索前预测器作为奖励模型，利用直接偏好优化（DPO）在查询重构任务上训练大型语言模型（LLM），从而展示了VQPP的适用性。我们在https://github.com/AdrianLutu/VQPP 发布了我们的基准和代码。