This paper extends the Isotonic Mechanism from the single-owner to multi-owner settings, in an effort to make it applicable to peer review where a paper often has multiple authors. Our approach starts by partitioning all submissions of a machine learning conference into disjoint blocks, each of which shares a common set of co-authors. We then employ the Isotonic Mechanism to elicit a ranking of the submissions from each author and to produce adjusted review scores that align with both the reported ranking and the original review scores. The generalized mechanism uses a weighted average of the adjusted scores on each block. We show that, under certain conditions, truth-telling by all authors is a Nash equilibrium for any valid partition of the overlapping ownership sets. However, we demonstrate that while the mechanism's performance in terms of estimation accuracy depends on the partition structure, optimizing this structure is computationally intractable in general. We develop a nearly linear-time greedy algorithm that provably finds a performant partition with appealing robust approximation guarantees. Extensive experiments on both synthetic data and real-world conference review data demonstrate the effectiveness of this generalized Isotonic Mechanism.
翻译:本文将 Isotonic 机制从单一所有者场景扩展至多所有者场景,旨在使其适用于论文常有多位作者的同行评议过程。我们的方法首先将机器学习会议的所有投稿划分为互不相交的区块,每个区块包含一组共同的合著者。随后采用 Isotonic 机制从每位作者处获取投稿排序,并生成与报告排序及原始评审分数均一致的调整后评分。该广义机制对每个区块的调整分数进行加权平均。研究表明,在特定条件下,对于重叠所有权集合的任何有效划分,所有作者的真实报告构成纳什均衡。然而我们证明,虽然该机制在估计精度上的表现取决于划分结构,但优化该结构在一般情况下计算上难以处理。我们提出一种近乎线性时间的贪心算法,能够可靠地找到具有鲁棒近似保证的高性能划分。在合成数据与真实会议评审数据上的大量实验验证了该广义 Isotonic 机制的有效性。