A major threat to the peer-review systems of computer science conferences is the existence of "collusion rings" between reviewers. In such collusion rings, reviewers who have also submitted their own papers to the conference work together to manipulate the conference's paper assignment, with the aim of being assigned to review each other's papers. The most straightforward way that colluding reviewers can manipulate the paper assignment is by indicating their interest in each other's papers through strategic paper bidding. One potential approach to solve this important problem would be to detect the colluding reviewers from their manipulated bids, after which the conference can take appropriate action. While prior work has developed effective techniques to detect other kinds of fraud, no research has yet established that detecting collusion rings is even possible. In this work, we tackle the question of whether it is feasible to detect collusion rings from the paper bidding. To answer this question, we conduct empirical analysis of two realistic conference bidding datasets, including evaluations of existing algorithms for fraud detection in other applications. We find that collusion rings can achieve considerable success at manipulating the paper assignment while remaining hidden from detection: for example, in one dataset, undetected colluders are able to achieve assignment to up to 30% of the papers authored by other colluders. In addition, when 10 colluders bid on all of each other's papers, no detection algorithm outputs a group of reviewers with more than 31% overlap with the true colluders. These results suggest that collusion cannot be effectively detected from the bidding using popular existing tools, demonstrating the need to develop more complex detection algorithms as well as those that leverage additional metadata (e.g., reviewer-paper text-similarity scores).

翻译：对计算机科学会议同行评审系统的主要威胁之一是审稿人之间存在的“合谋圈”。在这种合谋圈中，同样向会议提交了论文的审稿人合作操纵会议的论文分配，目的是被分配审阅彼此的论文。合谋审稿人操纵论文分配最直接的方式是通过策略性投标表达对彼此论文的兴趣。解决这一重要问题的一种潜在方法是从被操纵的投标中检测出合谋审稿人，之后会议可以采取适当行动。虽然先前的研究已开发出有效的技术来检测其他类型的欺诈行为，但尚无研究证实检测合谋圈是可能的。在本工作中，我们探讨了从论文投标中检测合谋圈的可行性问题。为回答此问题，我们对两个现实的会议投标数据集进行了实证分析，包括评估其他应用中用于欺诈检测的现有算法。我们发现合谋圈能够在隐藏自身不被检测的情况下成功操纵论文分配：例如，在一个数据集中，未被检测出的合谋者能够被分配到其他合谋者所写论文中高达30%的审稿任务。此外，当10名合谋者对彼此的所有论文进行投标时，没有任何检测算法能输出与真实合谋者重叠率超过31%的审稿人组。这些结果表明，使用流行的现有工具无法从投标中有效检测出合谋，这证明了开发更复杂的检测算法以及利用额外元数据（例如审稿人-论文文本相似度分数）的算法的必要性。