Neural ranking models (NRMs) have demonstrated effective performance in several information retrieval (IR) tasks. However, training NRMs often requires large-scale training data, which is difficult and expensive to obtain. To address this issue, one can train NRMs via weak supervision, where a large dataset is automatically generated using an existing ranking model (called the weak labeler) for training NRMs. Weakly supervised NRMs can generalize from the observed data and significantly outperform the weak labeler. This paper generalizes this idea through an iterative re-labeling process, demonstrating that weakly supervised models can iteratively play the role of weak labeler and significantly improve ranking performance without using manually labeled data. The proposed Generalized Weak Supervision (GWS) solution is generic and orthogonal to the ranking model architecture. This paper offers four implementations of GWS: self-labeling, cross-labeling, joint cross- and self-labeling, and greedy multi-labeling. GWS also benefits from a query importance weighting mechanism based on query performance prediction methods to reduce noise in the generated training data. We further draw a theoretical connection between self-labeling and Expectation-Maximization. Our experiments on two passage retrieval benchmarks suggest that all implementations of GWS lead to substantial improvements compared to weak supervision in all cases.

翻译：神经排序模型（Neural Ranking Models, NRMs）已在多项信息检索（Information Retrieval, IR）任务中展现出优异性能。然而，训练NRMs通常需要大规模训练数据，获取此类数据既困难且成本高昂。为解决这一问题，可通过弱监督方式训练NRMs，即利用现有排序模型（称为弱标注器）自动生成大规模数据集用于NRMs训练。弱监督训练的NRMs能够从观测数据中泛化，并显著超越弱标注器的性能。本文通过迭代重标注过程推广了这一思想，证明弱监督模型可迭代扮演弱标注器角色，在无需人工标注数据的情况下显著提升排序性能。所提出的广义弱监督（Generalized Weak Supervision, GWS）方案具有通用性，且与排序模型架构无关。本文给出了GWS的四种实现方式：自标注、交叉标注、联合交叉与自标注、以及贪心多标注。GWS还基于查询性能预测方法，利用查询重要性加权机制来减少生成训练数据中的噪声。我们进一步在理论上建立了自标注与期望最大化（Expectation-Maximization）之间的联系。在两个段落检索基准数据集上的实验表明，GWS的所有实现方式在所有情况下相比弱监督均能带来显著性能提升。