The assignment of papers to reviewers is a crucial part of the peer review processes of large publication venues, where organizers (e.g., conference program chairs) rely on algorithms to perform automated paper assignment. As such, a major challenge for the organizers of these processes is to specify paper assignment algorithms that find appropriate assignments with respect to various desiderata. Although the main objective when choosing a good paper assignment is to maximize the expertise of each reviewer for their assigned papers, several other considerations make introducing randomization into the paper assignment desirable: robustness to malicious behavior, the ability to evaluate alternative paper assignments, reviewer diversity, and reviewer anonymity. However, it is unclear in what way one should randomize the paper assignment in order to best satisfy all of these considerations simultaneously. In this work, we present a practical, one-size-fits-all method for randomized paper assignment intended to perform well across different motivations for randomness. We show theoretically and experimentally that our method outperforms currently-deployed methods for randomized paper assignment on several intuitive randomness metrics, demonstrating that the randomized assignments produced by our method are general-purpose.
翻译:论文与审稿人的分配是大型出版场所(例如会议程序主席依赖算法进行自动化论文分配)同行评审过程的关键环节。因此,这些流程的组织者面临的主要挑战是设计能够满足多种期望的论文分配算法。尽管选择优质论文分配的主要目标是最大化每位审稿人对其所分配论文的专业匹配度,但引入随机化机制还涉及其他考量:对恶意行为的鲁棒性、评估替代分配方案的能力、审稿人多样性以及审稿人匿名性。然而,如何以最佳方式实现论文分配的随机化以同时满足所有考量仍不明确。本研究提出了一种实用、普适的随机化论文分配方法,旨在在不同随机化动机下均表现优异。通过理论分析与实验验证,我们证明该方法在多种直观随机性指标上优于现行部署的随机化分配方案,表明该方法生成的随机化分配具有通用性。