The PLAID (Performance-optimized Late Interaction Driver) algorithm for ColBERTv2 uses clustered term representations to retrieve and progressively prune documents for final (exact) document scoring. In this paper, we reproduce and fill in missing gaps from the original work. By studying the parameters PLAID introduces, we find that its Pareto frontier is formed of a careful balance among its three parameters; deviations beyond the suggested settings can substantially increase latency without necessarily improving its effectiveness. We then compare PLAID with an important baseline missing from the paper: re-ranking a lexical system. We find that applying ColBERTv2 as a re-ranker atop an initial pool of BM25 results provides better efficiency-effectiveness trade-offs in low-latency settings. However, re-ranking cannot reach peak effectiveness at higher latency settings due to limitations in recall of lexical matching and provides a poor approximation of an exhaustive ColBERTv2 search. We find that recently proposed modifications to re-ranking that pull in the neighbors of top-scoring documents overcome this limitation, providing a Pareto frontier across all operational points for ColBERTv2 when evaluated using a well-annotated dataset. Curious about why re-ranking methods are highly competitive with PLAID, we analyze the token representation clusters PLAID uses for retrieval and find that most clusters are predominantly aligned with a single token and vice versa. Given the competitive trade-offs that re-ranking baselines exhibit, this work highlights the importance of carefully selecting pertinent baselines when evaluating the efficiency of retrieval engines.

翻译：PLAID（性能优化延迟交互驱动）算法用于ColBERTv2，通过聚类词元表征检索并逐步裁剪文档以实现最终（精确）文档评分。本文在复现原始工作的同时填补其缺失环节。通过研究PLAID引入的参数，我们发现其帕累托前沿由三个参数间的审慎平衡构成；偏离建议设置可能显著增加延迟而不一定提升有效性。随后，我们将PLAID与论文中缺失的重要基线进行对比：对词汇系统进行重排序。研究发现，在BM25初始结果集上应用ColBERTv2作为重排序器，能够在低延迟场景下提供更优的效率-有效性权衡。然而，受限于词汇匹配的召回率，重排序在高延迟设置下无法达到峰值有效性，且只能粗略近似穷举式ColBERTv2搜索。最新提出的重排序改进方法（通过拉取高分文档的邻近结果）克服了这一局限，在使用高质量标注数据集评估时，为ColBERTv2提供了覆盖所有操作点的帕累托前沿。为探究重排序方法为何能与PLAID竞争，我们分析了PLAID用于检索的词元表征聚类，发现多数聚类与单一词元高度对齐，反之亦然。鉴于重排序基线展现出的竞争性权衡，本研究凸显了在评估检索引擎效率时审慎选择相关基线的关键性。