Retrieving external knowledge is essential for solving real-world tasks, yet it remains challenging when the relationship between a query and its relevant knowledge involves implicit and complex reasoning beyond surface-level semantic or lexical matching (e.g., mathematical problems relying on the same theorem or coding requiring deep reasoning). Existing approaches primarily rely on query-side reasoning (e.g., query rewriting), which introduces significant online latency and underutilizes the opportunity to perform reasoning over the knowledge corpus itself (i.e., index-side reasoning). In this paper, we propose RL-Index, an agentic indexing framework that formulates retrieval index reasoning as a reinforcement learning problem. Instead of performing reasoning at query time, RL-Index shifts reasoning to the indexing stage by augmenting documents with LLM-generated rationales that explicitly encode the latent query-knowledge relationship. To optimize the quality of these rationales, we employ Group Relative Policy Optimization (GRPO) and use retrieval similarity as a verifiable reward signal, enabling direct optimization of indexing decisions for retrieval effectiveness. Extensive experiments on the BRIGHT benchmark demonstrate that RL-Index consistently improves both retrieval and downstream question-answering performance, while significantly reducing online inference latency. Moreover, the learned rationale augmentation generalizes across diverse retrievers and generators, highlighting its robustness as a plug-and-play indexing strategy across different retrieval systems.

翻译：检索外部知识对解决现实任务至关重要，但查询与其相关知识之间的关系往往涉及隐式复杂推理（例如依赖相同定理的数学问题或需要深层推理的编码任务），而现有方法主要依赖查询端推理（如查询重写），这不仅会引入显著的在线延迟，还可能利用推理知识语料库本身（即索引端推理）的机会，从而未能充分挖掘其潜力。本文提出RL-Index，一种将检索索引推理形式化为强化学习问题的智能索引框架。该框架将推理过程从查询时迁移至索引阶段，通过用大语言模型生成的原理增强文档，显式编码潜在查询-知识关系。为优化这些原理的质量，我们采用群组相对策略优化，并以检索相似度作为可验证的奖励信号，直接优化索引决策以提升检索效果。在BRIGHT基准上的实验表明，RL-Index持续改善检索与下游问答性能，同时显著降低在线推理延迟。此外，学习到的原理增强策略可泛化至不同的检索器与生成器，凸显其作为即插即用型索引策略在不同检索系统中的鲁棒性。