Little is known whether peer reviewers use the same evaluation criteria and how they integrate the criteria into their overall judgment. This study therefore proposed two assessment styles based on theoretical perspectives and normative positions. According to the case-by-case style, referees use many and different criteria, weight criteria on a case-by-case basis, and integrate criteria in a complex, non-mechanical way into their overall judgment. According to the uniform style, referees use a small fraction of the available criteria, apply the same criteria, weight the criteria in the same way, and integrate the criteria based on simple rules (i.e., fast-and-frugal heuristics). These two styles were examined using a unique dataset from a career funding scheme that contained a comparatively large number of evaluation criteria. A heuristic (fast-and-frugal trees) and a complex procedure (logistic regression) were employed to describe how referees integrate the criteria into their overall judgment. The logistic regression predicted the referees' overall assessment with high accuracy and slightly more accurately than the fast-and-frugal trees. Overall, the results of this study support the uniform style but also indicate that the uniform style needs to be revised as follows: referees use many criteria and integrate the criteria using complex rules. However, and most importantly, the revised style could describe most - but not all - of the referees' judgments. Future studies should therefore examine how referees' judgments can be characterized in those cases where the uniform style failed. Moreover, the evaluation process of referees should be studied in more empirical and theoretical detail.

翻译：关于同行评审者是否使用相同的评价标准以及如何将这些标准整合到其整体判断中，目前知之甚少。因此，本研究基于理论视角和规范性立场提出了两种评估风格。根据个案评估风格，评审者使用多种不同的标准，对个案进行权重分配，并以复杂、非机械的方式将这些标准整合到整体判断中。根据统一评估风格，评审者仅使用可用标准的一小部分，应用相同的标准，以相同方式分配权重，并基于简单规则（即快速节俭启发式）整合标准。利用一个包含相对较多评价指标的职业生涯资助计划的独特数据集，我们对这两种风格进行了检验。采用启发式（快速节俭树）和复杂程序（逻辑回归）来描述评审者如何将标准整合到整体判断中。逻辑回归对评审者整体评估的预测准确度很高，且略高于快速节俭树。总体而言，本研究结果支持统一评估风格，但也表明统一评估风格需要修订如下：评审者使用多种标准，并采用复杂规则整合标准。然而，最重要的是，修订后的风格能描述大多数（但非全部）评审者的判断。因此，未来研究应考察在统一风格失效的情况下，评审者的判断如何被特征化。此外，评审者的评估过程还需在更实证和理论的细节层面进行深入研究。