Machine learning and artificial intelligence conferences such as NeurIPS and ICML now regularly receive tens of thousands of submissions, posing significant challenges to maintaining the quality and consistency of the peer review process. This challenge is particularly acute for best paper awards, which are an important part of the peer review process, yet whose selection has increasingly become a subject of debate in recent years. In this paper, we introduce an author-assisted mechanism to facilitate the selection of best paper awards. Our method employs the Isotonic Mechanism for eliciting authors' assessments of their own submissions in the form of a ranking, which is subsequently utilized to adjust the raw review scores for optimal estimation of the submissions' ground-truth quality. We demonstrate that authors are incentivized to report truthfully when their utility is a convex additive function of the adjusted scores, and we validate this convexity assumption for best paper awards using publicly accessible review data of ICLR from 2019 to 2023 and NeurIPS from 2021 to 2023. Crucially, in the special case where an author has a single quota -- that is, may nominate only one paper -- we prove that truthfulness holds even when the utility function is merely nondecreasing and additive. This finding represents a substantial relaxation of the assumptions required in prior work. For practical implementation, we extend our mechanism to accommodate the common scenario of overlapping authorship. Finally, simulation results demonstrate that our mechanism significantly improves the quality of papers selected for awards.

翻译：如今，NeurIPS和ICML等机器学习与人工智能会议每年收到数以万计的投稿，这对维持同行评审过程的质量与一致性构成了重大挑战。这一挑战在最佳论文奖的评选上尤为突出：最佳论文奖是同行评审过程的重要组成部分，但其评选方式近年来日益成为争议的焦点。本文提出一种作者辅助机制，以促进最佳论文奖的评选。我们的方法采用等渗机制，以排序形式获取作者对其自身投稿的评估，并利用该评估调整原始评审分数，从而实现对投稿真实质量的最优估计。我们证明，当作者的效用函数是调整后分数的凸可加函数时，作者有动机如实报告；我们利用2019年至2023年ICLR以及2021年至2023年NeurIPS公开可获取的评审数据，验证了最佳论文奖评选场景下该凸性假设的合理性。关键的是，在作者仅拥有单一配额（即只能提名一篇论文）的特殊情况下，我们证明即使效用函数仅为非递减可加函数，真实性依然成立。这一发现显著放宽了先前研究所需的假设条件。为便于实际应用，我们将该机制扩展至处理常见的作者重叠情形。最终的仿真结果表明，我们的机制能显著提升获奖论文的遴选质量。