Information retrieval methods often rely on a single embedding model trained on large, general-domain datasets like MSMARCO. While this approach can produce a retriever with reasonable overall performance, they often underperform models trained on domain-specific data when testing on their respective domains. Prior work in information retrieval has tackled this through multi-task training, but the idea of routing over a mixture of domain-specific expert retrievers remains unexplored despite the popularity of such ideas in language model generation research. In this work, we introduce RouterRetriever, a retrieval model that leverages a mixture of domain-specific experts by using a routing mechanism to select the most appropriate expert for each query. RouterRetriever is lightweight and allows easy addition or removal of experts without additional training. Evaluation on the BEIR benchmark demonstrates that RouterRetriever outperforms both models trained on MSMARCO (+2.1 absolute nDCG@10) and multi-task models (+3.2). This is achieved by employing our routing mechanism, which surpasses other routing techniques (+1.8 on average) commonly used in language modeling. Furthermore, the benefit generalizes well to other datasets, even in the absence of a specific expert on the dataset. RouterRetriever is the first work to demonstrate the advantages of routing over a mixture of domain-specific expert embedding models as an alternative to a single, general-purpose embedding model, especially when retrieving from diverse, specialized domains.

翻译：信息检索方法通常依赖于在MSMARCO等大规模通用领域数据集上训练的单一嵌入模型。尽管这种方法能产生整体性能尚可的检索器，但在各自特定领域测试时，其表现往往逊色于针对该领域数据训练的模型。先前的信息检索研究通过多任务训练应对此问题，然而，尽管此类思想在语言模型生成研究中颇为流行，基于领域特定专家检索器混合进行路由的设想仍未得到探索。本文提出RouterRetriever，这是一种通过路由机制为每个查询选择最合适领域专家的检索模型，其利用领域特定专家混合进行检索。RouterRetriever结构轻量，支持无需额外训练即可灵活增删专家模块。在BEIR基准测试上的评估表明，RouterRetriever显著优于基于MSMARCO训练的模型（nDCG@10绝对值提升+2.1）及多任务模型（提升+3.2）。这一成果得益于我们设计的路由机制，该机制平均优于语言建模中常用的其他路由技术（+1.8）。此外，即使面对缺乏对应领域专家的数据集，该优势仍能良好泛化至其他数据集。RouterRetriever首次论证了：相较于单一通用嵌入模型，采用领域特定专家嵌入模型混合路由方案在检索多样化专业领域信息时具有显著优势。