Peer-evaluation and selection systems are used when sets of agents evaluate each other in order to select the best $k$ among them. These are commonly used in real-world settings, including academic conferences where those reviewing papers are often the set of submitters. Conferences have attempted to better allocate their reviewing resources by moving to a two-stage mechanism, in which some papers are eliminated after a first stage of review and remaining papers receive additional reviewers. We investigate how two major strategyproof peer selection mechanisms, Partition and ExactDollarPartition, perform when adapted to a two-stage system, in order to try and understand the effect of the two-stage mechanism on which agents get selected. We also examine how the various parameters of the two-stage mechanism influence the outcome. We provide a theoretical basis by showing how a particular setting is influenced by the two stages. However, solving for the general case seems implausible at the moment, and we use extensive simulations of different scenarios and settings to observe which agents benefit and which are harmed by adopting two-stage mechanisms (and we vary this mechanisms parameters as well). We show that the two-stage mechanism's advantage depends the noisiness of reviewer beliefs. Borderline agents benefit most in a low noise environment, while high rank agents benefit more in noisy environments. We show that the effectiveness of these mechanisms is highly dependent on the number of chosen agents, the number of reviews requested from agents, and reviewers' correlation, indicating that organizers need to exercise caution when selecting these parameters for a reviewing process.

翻译：同行评审与筛选系统广泛应用于由一组评审者相互评估以选出最佳$k$个对象的情境中。这类系统常见于现实场景，例如学术会议中论文评审者往往就是投稿者本身。为优化评审资源分配，部分会议已转向两阶段评审机制：第一阶段淘汰部分论文后，剩余论文将获得更多评审者。本文探究两种策略性防策略操纵的同行评审机制——分区法与精确美元分区法——在两阶段系统中的表现，旨在理解两阶段机制对入选对象的影响。我们同时检验该机制中不同参数对结果的调节作用。通过展示特定场景受两阶段影响的理论基础，我们揭示了其作用机理。然而由于通用求解当前不可行，我们采用多场景多参数的大规模仿真，观察不同参与者在两阶段机制（含参数变体）下的获益与受损情况。研究表明：两阶段机制的优势取决于评审者信念的噪声水平。在低噪声环境下，边缘候选者获益最大；而在高噪声环境中，高排名候选者更具优势。这些机制的有效性高度依赖于被选对象数量、每位评审者所需完成的评审数以及评审者间的相关性，提示组织者在选择评审流程参数时需谨慎决策。