The assignment of papers to reviewers is a crucial part of the peer review processes of large publication venues, where organizers (e.g., conference program chairs) rely on algorithms to perform automated paper assignment. As such, a major challenge for the organizers of these processes is to specify paper assignment algorithms that find appropriate assignments with respect to various desiderata. Although the main objective when choosing a good paper assignment is to maximize the expertise of each reviewer for their assigned papers, several other considerations make introducing randomization into the paper assignment desirable: robustness to malicious behavior, the ability to evaluate alternative paper assignments, reviewer diversity, and reviewer anonymity. However, it is unclear in what way one should randomize the paper assignment in order to best satisfy all of these considerations simultaneously. In this work, we present a practical, one-size-fits-all method for randomized paper assignment intended to perform well across different motivations for randomness. We show theoretically and experimentally that our method outperforms currently-deployed methods for randomized paper assignment on several intuitive randomness metrics, demonstrating that the randomized assignments produced by our method are general-purpose.
翻译:论文分配给审稿人是大型出版场所同行评审过程中的关键环节,组织者(例如会议程序主席)依赖算法来执行自动化的论文分配。因此,这些过程的组织者面临的主要挑战是指定论文分配算法,以根据各种期望找到合适的分配。虽然选择良好的论文分配时的主要目标是最大化每位审稿人在其分配论文上的专业知识,但其他几个考虑因素使得在论文分配中引入随机化变得可取:对恶意行为的鲁棒性、评估替代论文分配的能力、审稿人多样性和审稿人匿名性。然而,尚不清楚应如何对论文分配进行随机化,以同时最好地满足所有这些考虑因素。在这项工作中,我们提出了一种实用的、通用的随机化论文分配方法,旨在针对随机性的不同动机表现出良好性能。我们从理论和实验上证明,在几个直观的随机性度量上,我们的方法优于当前部署的随机化论文分配方法,表明我们的方法产生的随机化分配具有通用性。