The first-stage retrieval aims to retrieve a subset of candidate documents from a huge collection both effectively and efficiently. Since various matching patterns can exist between queries and relevant documents, previous work tries to combine multiple retrieval models to find as many relevant results as possible. The constructed ensembles, whether learned independently or jointly, do not care which component model is more suitable to an instance during training. Thus, they cannot fully exploit the capabilities of different types of retrieval models in identifying diverse relevance patterns. Motivated by this observation, in this paper, we propose a Mixture-of-Experts (MoE) model consisting of representative matching experts and a novel competitive learning mechanism to let the experts develop and enhance their expertise during training. Specifically, our MoE model shares the bottom layers to learn common semantic representations and uses differently structured upper layers to represent various types of retrieval experts. Our competitive learning mechanism has two stages: (1) a standardized learning stage to train the experts equally to develop their capabilities to conduct relevance matching; (2) a specialized learning stage where the experts compete with each other on every training instance and get rewards and updates according to their performance to enhance their expertise on certain types of samples. Experimental results on three retrieval benchmark datasets show that our method significantly outperforms the state-of-the-art baselines.

翻译：第一阶段检索旨在从海量文档集合中高效且有效地检索出候选文档子集。由于查询与相关文档之间可能存在多种匹配模式，以往的工作尝试组合多种检索模型以尽可能多地发现相关结果。无论是独立学习还是联合学习，所构建的集成模型在训练过程中并不关心哪个组件模型更适合某个实例。因此，它们无法充分利用不同类型检索模型在识别多样化相关性模式方面的能力。基于这一观察，本文提出了一种混合专家（MoE）模型，该模型由具有代表性的匹配专家和一种新颖的竞争学习机制组成，使专家能够在训练过程中发展和增强其专业能力。具体而言，我们的MoE模型共享底层以学习通用语义表示，并使用不同结构的上层来表示不同类型的检索专家。我们的竞争学习机制分为两个阶段：（1）标准化学习阶段，平等训练专家以发展其进行相关性匹配的能力；（2）专门化学习阶段，专家在每个训练实例上相互竞争，并根据其表现获得奖励和更新，以增强其在特定类型样本上的专业能力。在三个检索基准数据集上的实验结果表明，我们的方法显著优于最先进的基线模型。