Parameter-Efficient transfer learning with Adapters have been studied in Natural Language Processing (NLP) as an alternative to full fine-tuning. Adapters are memory-efficient and scale well with downstream tasks by training small bottle-neck layers added between transformer layers while keeping the large pretrained language model (PLMs) frozen. In spite of showing promising results in NLP, these methods are under-explored in Information Retrieval. While previous studies have only experimented with dense retriever or in a cross lingual retrieval scenario, in this paper we aim to complete the picture on the use of adapters in IR. First, we study adapters for SPLADE, a sparse retriever, for which adapters not only retain the efficiency and effectiveness otherwise achieved by finetuning, but are memory-efficient and orders of magnitude lighter to train. We observe that Adapters-SPLADE not only optimizes just 2\% of training parameters, but outperforms fully fine-tuned counterpart and existing parameter-efficient dense IR models on IR benchmark datasets. Secondly, we address domain adaptation of neural retrieval thanks to adapters on cross-domain BEIR datasets and TripClick. Finally, we also consider knowledge sharing between rerankers and first stage rankers. Overall, our study complete the examination of adapters for neural IR

翻译：参数高效的适配器迁移学习已在自然语言处理领域作为全参数微调的替代方案得到研究。适配器通过保持大型预训练语言模型冻结，在Transformer层间添加并训练小型瓶颈层，具有内存高效性且能良好适应下游任务。尽管这些方法在NLP中展现出显著效果，但在信息检索领域尚未得到充分探索。鉴于以往研究仅涉及稠密检索器或跨语言检索场景，本文旨在全面探究适配器在IR中的应用。首先，我们研究了针对稀疏检索器SPLADE的适配器：适配器不仅保留了微调所能实现的效率与效果，更实现了内存高效性且训练量级降低数个数量级。实验表明，Adapter-SPLADE不仅仅优化了2%的训练参数，更在IR基准数据集上超越了全参数微调版本及现有参数高效稠密IR模型。其次，我们通过跨域BEIR数据集与TripClick上的适配器技术，解决了神经检索的领域适配问题。最后，我们还探讨了重排序器与初排序器之间的知识共享。总体而言，本研究完整检验了适配器在神经IR中的应用。