Existing cross-encoder re-rankers can be categorized as pointwise, pairwise, or listwise models. Pair- and listwise models allow passage interactions, which usually makes them more effective than pointwise models but also less efficient and less robust to input order permutations. To enable efficient permutation-invariant passage interactions during re-ranking, we propose a new cross-encoder architecture with inter-passage attention: the Set-Encoder. In Cranfield-style experiments on TREC Deep Learning and TIREx, the Set-Encoder is as effective as state-of-the-art listwise models while improving efficiency and robustness to input permutations. Interestingly, a pointwise model is similarly effective, but when additionally requiring the models to consider novelty, the Set-Encoder is more effective than its pointwise counterpart and retains its advantageous properties compared to other listwise models. Our code and models are publicly available at https://github.com/webis-de/set-encoder.

翻译：现有的跨编码器重排模型可分为逐点式、成对式或列表式模型。成对式和列表式模型允许段落间交互，这通常使其比逐点式模型更有效，但效率较低且对输入顺序排列的鲁棒性较弱。为实现重排过程中高效且排列不变的段落交互，我们提出了一种具有段落间注意力的新型跨编码器架构：Set-Encoder。在TREC深度学习和TIREx数据集上的克兰菲尔德风格实验中，Set-Encoder在保持与先进列表式模型相当效能的同时，提升了效率及对输入排列的鲁棒性。有趣的是，逐点式模型具有类似效能，但当额外要求模型考虑新颖性时，Set-Encoder较其逐点式对应模型更为有效，并相较于其他列表式模型保留了其优势特性。我们的代码与模型已在https://github.com/webis-de/set-encoder公开。