Cross-encoders deliver state-of-the-art ranking effectiveness in information retrieval, but have a high inference cost. This prevents them from being used as first-stage rankers, but also incurs a cost when re-ranking documents. Prior work has addressed this bottleneck from two largely separate directions: accelerating cross-encoder inference by sparsifying the attention process or improving first-stage retrieval effectiveness using more complex models, e.g. late-interaction ones. In this work, we propose to bridge these two approaches, based on an in-depth understanding of the internal mechanisms of cross-encoders. Starting from cross-encoders, we show that it is possible to derive a new late-interaction-like architecture by carefully removing detrimental or unnecessary interactions. We name this architecture MICE (Minimal Interaction Cross-Encoders). We extensively evaluate MICE across both in-domain (ID) and out-of-domain (OOD) datasets. MICE decreases fourfold the inference latency compared to standard cross-encoders, matching late-interaction models like ColBERT while retaining most of cross-encoder ID effectiveness and demonstrating superior generalization abilities in OOD.

翻译：交叉编码器在信息检索中实现了最先进的排序效果，但具有较高的推理成本。这阻碍了其作为首阶段排序器的应用，同时在重排序文档时也会产生显著开销。先前的研究主要从两个相对独立的方向解决这一瓶颈：通过稀疏化注意力机制来加速交叉编码器推理，或使用更复杂的模型（例如延迟交互模型）提升首阶段检索效果。本文基于对交叉编码器内部机制的深入理解，提出将这两种方法相结合。从交叉编码器出发，我们证明通过精心移除有害或不必要的交互，可以推导出一种新的类延迟交互架构。我们将此架构命名为MICE（最小交互交叉编码器）。我们在领域内（ID）和领域外（OOD）数据集上对MICE进行了全面评估。与标准交叉编码器相比，MICE将推理延迟降低至四分之一，在保持交叉编码器大部分领域内有效性的同时，其性能与ColBERT等延迟交互模型相当，并在领域外场景中展现出更优异的泛化能力。