Retrieval-augmented generation (RAG) systems address complex user requests by decomposing them into subqueries, retrieving potentially relevant documents for each, and then aggregating them to generate an answer. Efficiently selecting informative documents requires balancing a key trade-off: (i) retrieving broadly enough to capture all the relevant material, and (ii) limiting retrieval to avoid excessive noise and computational cost. We formulate query decomposition and document retrieval in an exploitation-exploration setting, where retrieving one document at a time builds a belief about the utility of a given sub-query and informs the decision to continue exploiting or exploring an alternative. We experiment with a variety of bandit learning methods and demonstrate their effectiveness in dynamically selecting the most informative sub-queries. Our main finding is that estimating document relevance using rank information and human judgments yields a 35% gain in document-level precision, 15% increase in {\alpha}-nDCG, and better performance on the downstream task of long-form generation.

翻译：检索增强生成（RAG）系统通过将复杂的用户请求分解为子查询、为每个子查询检索潜在相关文档并聚合结果以生成答案。高效选择信息丰富的文档需要权衡一个关键问题：（i）检索范围需足够广泛以覆盖所有相关材料，同时（ii）需限制检索以避免过度噪声和计算成本。我们将查询分解与文档检索建模为利用-探索框架，其中每次检索单个文档会形成对特定子查询效用的信念，并据此决定继续利用当前路径或探索替代方案。我们通过多种赌博机学习方法进行实验，证明其能动态选择最具信息量的子查询。主要发现是：利用排序信息与人工标注估计文档相关性，可使文档级精确率提升35%、α-nDCG提高15%，并在长文本生成的下游任务中取得更优性能。