Conversational search, unlike single-turn retrieval tasks, requires understanding the current question within a dialogue context. The common approach of rewrite-then-retrieve aims to decontextualize questions to be self-sufficient for off-the-shelf retrievers, but most existing methods produce sub-optimal query rewrites due to the limited ability to incorporate signals from the retrieval results. To overcome this limitation, we present a novel framework RetPO (Retriever's Preference Optimization), which is designed to optimize a language model (LM) for reformulating search queries in line with the preferences of the target retrieval systems. The process begins by prompting a large LM to produce various potential rewrites and then collects retrieval performance for these rewrites as the retrievers' preferences. Through the process, we construct a large-scale dataset called RF collection, containing Retrievers' Feedback on over 410K query rewrites across 12K conversations. Furthermore, we fine-tune a smaller LM using this dataset to align it with the retrievers' preferences as feedback. The resulting model achieves state-of-the-art performance on two recent conversational search benchmarks, significantly outperforming existing baselines, including GPT-3.5.

翻译：对话式搜索不同于单轮检索任务，需要理解对话上下文中的当前问题。常见的“改写后检索”方法旨在将问题去语境化，使其能够独立适用于现成检索器，但现有方法大多由于难以有效融入检索结果的反馈信号，导致查询改写结果次优。为解决这一局限，我们提出新型框架RetPO（检索器偏好优化），该框架旨在优化语言模型，使其按照目标检索系统的偏好重构搜索查询。该流程首先促使大型语言模型生成多种潜在改写，随后收集这些改写在检索器上的表现作为检索器偏好。通过这一过程，我们构建了名为RF collection的大规模数据集，包含12K个对话中超过41万条查询改写的检索器反馈。进一步地，我们利用该数据集微调一个小型语言模型，使其与作为反馈的检索器偏好对齐。最终模型在两个最新对话式搜索基准测试中达到当前最优水平，显著超越包括GPT-3.5在内的现有基线方法。