Unlike traditional fact-based retrieval, rationale-based retrieval typically necessitates cross-encoding of query-document pairs using large language models, incurring substantial computational costs. To address this limitation, we propose Rabtriever, which independently encodes queries and documents, while providing comparable cross query-document comprehension capabilities to rerankers. We start from training a LLM-based generative reranker, which puts the document prior to the query and prompts the LLM to generate the relevance score by log probabilities. We then employ it as the teacher of an on-policy distillation framework, with Rabtriever as the student to reconstruct the teacher's contextual-aware query embedding. To achieve this effect, Rabtriever is first initialized from the teacher, with parameters frozen. The Joint-Embedding Predictive Architecture (JEPA) paradigm is then adopted, which integrates a lightweight, trainable predictor between LLM layers and heads, projecting the query embedding into a new hidden space, with the document embedding as the latent vector. JEPA then minimizes the distribution difference between this projected embedding and the teacher embedding. To strengthen the sampling efficiency of on-policy distillation, we also add an auxiliary loss on the reverse KL of LLM logits, to reshape the student's logit distribution. Rabtriever optimizes the teacher's quadratic complexity on the document length to linear, verified both theoretically and empirically. Experiments show that Rabtriever outperforms different retriever baselines across diverse rationale-based tasks, including empathetic conversations and robotic manipulations, with minor accuracy degradation from the reranker. Rabtriever also generalizes well on traditional retrieval benchmarks such as MS MARCO and BEIR, with comparable performance to the best retriever baseline.

翻译：与传统基于事实的检索不同，基于原理解析的检索通常需利用大语言模型对查询-文档对进行交叉编码，导致高昂的计算成本。为解决此局限，我们提出RABTRIEVER，该模型独立编码查询与文档，同时具备与重排器相当的跨查询-文档理解能力。我们首先训练基于LLM的生成式重排器，将文档置于查询之前，通过对数概率驱动LLM生成相关性分数。随后将其作为在线策略蒸馏框架的教师模型，以RABTRIEVER为学生模型，重构教师的上下文感知查询嵌入。为实现此效果，RABTRIEVER首先从教师模型初始化并冻结参数，继而采用联合嵌入预测架构（JEPA）范式，在LLM层与层头间集成轻量可训练预测器，将查询嵌入投影至新隐空间，以文档嵌入作为潜向量。JEPA通过最小化该投影嵌入与教师嵌入之间的分布差异实现优化。为增强在线策略蒸馏的采样效率，我们额外引入基于LLM逻辑逆KL散度的辅助损失，重塑学生模型的逻辑分布。RABTRIEVER将教师在文档长度上的二次复杂度优化为线性复杂度，该特性在理论与实证中均得到验证。实验表明，RABTRIEVER在涵盖共情对话与机器人操控的多样化原理解析任务中，以轻微精度损失为代价，显著优于各类检索基线模型。同时，RABTRIEVER在MS MARCO与BEIR等传统检索基准上展现良好泛化能力，性能与最优检索基线相当。