The retrieval model is an indispensable component for real-world knowledge-intensive tasks, e.g., open-domain question answering (ODQA). As separate retrieval skills are annotated for different datasets, recent work focuses on customized methods, limiting the model transferability and scalability. In this work, we propose a modular retriever where individual modules correspond to key skills that can be reused across datasets. Our approach supports flexible skill configurations based on the target domain to boost performance. To mitigate task interference, we design a novel modularization parameterization inspired by sparse Transformer. We demonstrate that our model can benefit from self-supervised pretraining on Wikipedia and fine-tuning using multiple ODQA datasets, both in a multi-task fashion. Our approach outperforms recent self-supervised retrievers in zero-shot evaluations and achieves state-of-the-art fine-tuned retrieval performance on NQ, HotpotQA and OTT-QA.

翻译：检索模型是真实世界中知识密集型任务（例如开放域问答）不可或缺的组成部分。由于不同数据集标注了不同的独立检索技能，近期研究侧重于定制化方法，这限制了模型的迁移性与可扩展性。本文提出一种模块化检索器，其各模块对应可在数据集间复用的关键技能。我们的方法支持基于目标域灵活配置技能以提升性能。为缓解任务干扰，受稀疏Transformer启发，我们设计了一种新颖的模块化参数化方法。实验表明，通过维基百科上的自监督预训练与多任务微调多个开放域问答数据集，模型均能受益。在零样本评估中，我们的方法优于近期自监督检索器，并在NQ、HotpotQA和OTT-QA上实现了微调后的最优检索性能。