In the setting of conference peer review, the conference aims to accept high-quality papers and reject low-quality papers based on noisy review scores. A recent work proposes the isotonic mechanism, which can elicit the ranking of paper qualities from an author with multiple submissions to help improve the conference's decisions. However, the isotonic mechanism relies on the assumption that the author's utility is both an increasing and a convex function with respect to the review score, which is often violated in peer review settings (e.g.~when authors aim to maximize the number of accepted papers). In this paper, we propose a sequential review mechanism that can truthfully elicit the ranking information from authors while only assuming the agent's utility is increasing with respect to the true quality of her accepted papers. The key idea is to review the papers of an author in a sequence based on the provided ranking and conditioning the review of the next paper on the review scores of the previous papers. Advantages of the sequential review mechanism include 1) eliciting truthful ranking information in a more realistic setting than prior work; 2) improving the quality of accepted papers, reducing the reviewing workload and increasing the average quality of papers being reviewed; 3) incentivizing authors to write fewer papers of higher quality.
翻译:在会议同行评审的设定中,会议旨在基于带噪声的评审分数接收高质量论文并拒绝低质量论文。近期研究提出了等渗机制,可从提交多篇论文的作者处获取论文质量排序,以帮助改进会议的决策。然而,等渗机制依赖于作者效用函数关于评审分数既是递增又是凸函数的假设,这在同行评审设定中常被违反(例如当作者以最大化被接收论文数量为目标时)。本文提出一种顺序审稿机制,在仅假设代理人的效用关于其被接收论文的真实质量递增的条件下,即可从作者处诚实地获取排序信息。关键思想是基于作者提供的排序按顺序评审其论文,并根据前序论文的评审分数决定下一论文的评审。顺序审稿机制的优势包括:1)在比现有工作更现实的设定下获取真实排序信息;2)提升被接收论文的质量,减少审稿工作量并提高被评审论文的平均质量;3)激励作者撰写更少但质量更高的论文。