Embedding-based Retrieval Models (ERMs) have emerged as a promising framework for large-scale text retrieval problems due to powerful large language models. Nevertheless, fine-tuning ERMs to reach state-of-the-art results can be expensive due to the extreme scale of data as well as the complexity of multi-stages pipelines (e.g., pre-training, fine-tuning, distillation). In this work, we propose the PEFA framework, namely ParamEter-Free Adapters, for fast tuning of ERMs without any backward pass in the optimization. At index building stage, PEFA equips the ERM with a non-parametric k-nearest neighbor (kNN) component. At inference stage, PEFA performs a convex combination of two scoring functions, one from the ERM and the other from the kNN. Based on the neighborhood definition, PEFA framework induces two realizations, namely PEFA-XL (i.e., extra large) using double ANN indices and PEFA-XS (i.e., extra small) using a single ANN index. Empirically, PEFA achieves significant improvement on two retrieval applications. For document retrieval, regarding Recall@100 metric, PEFA improves not only pre-trained ERMs on Trivia-QA by an average of 13.2%, but also fine-tuned ERMs on NQ-320K by an average of 5.5%, respectively. For product search, PEFA improves the Recall@100 of the fine-tuned ERMs by an average of 5.3% and 14.5%, for PEFA-XS and PEFA-XL, respectively. Our code is available at https://github.com/ amzn/pecos/tree/mainline/examples/pefa-wsdm24

翻译：基于嵌入的检索模型（ERMs）凭借强大的大语言模型，已成为大规模文本检索问题中颇具前景的框架。然而，由于数据的极端规模以及多阶段流水线（如预训练、微调、蒸馏）的复杂性，微调ERMs以达到最先进结果可能代价高昂。本文提出PEFA框架，即无参数适配器，可在优化过程中无需任何反向传播即可快速调优ERMs。在索引构建阶段，PEFA为ERM配备非参数化的k近邻（kNN）组件。在推理阶段，PEFA对来自ERM和kNN的两个评分函数进行凸组合。基于邻域定义，PEFA框架衍生出两种实现：PEFA-XL（超大规模）使用双ANN索引，PEFA-XS（超小规模）使用单ANN索引。实验表明，PEFA在两个检索任务上均取得显著提升。在文档检索方面，以Recall@100为指标，PEFA不仅使Trivia-QA数据集上预训练ERM平均提升13.2%，还在NQ-320K数据集上使微调ERM平均提升5.5%。在产品搜索方面，PEFA-XS和PEFA-XL分别使微调ERM的Recall@100平均提升5.3%和14.5%。我们的代码开源在 https://github.com/amzn/pecos/tree/mainline/examples/pefa-wsdm24。