Model distillation has emerged as a prominent technique to improve neural search models. To date, distillation taken an offline approach, wherein a new neural model is trained to predict relevance scores between arbitrary queries and documents. In this paper, we explore a departure from this offline distillation strategy by investigating whether a model for a specific query can be effectively distilled from neural re-ranking results (i.e., distilling in an online setting). Indeed, we find that a lexical model distilled online can reasonably replicate the re-ranking of a neural model. More importantly, these models can be used as queries that execute efficiently on indexes. This second retrieval stage can enrich the pool of documents for re-ranking by identifying documents that were missed in the first retrieval stage. Empirically, we show that this approach performs favourably when compared with established pseudo relevance feedback techniques, dense retrieval methods, and sparse-dense ensemble "hybrid" approaches.

翻译：模型蒸馏已成为提升神经搜索模型的重要技术。迄今为止，蒸馏采用离线方式，即训练新的神经模型来预测任意查询与文档之间的相关性分数。本文探讨了对这种离线蒸馏策略的偏离，通过研究特定查询的模型能否有效地从神经重排序结果中蒸馏出来（即在线环境下的蒸馏）。事实上，我们发现在线蒸馏的词汇模型能够合理地重现神经模型的重排序结果。更重要的是，这些模型可以作为查询在索引上高效执行。这一第二检索阶段能够通过识别第一检索阶段遗漏的文档来丰富重排序的文档池。实验表明，与成熟的伪相关反馈技术、稠密检索方法以及稀疏-稠密集成“混合”方法相比，本方法表现优越。