In higher education courses, peer assessment activities are common for keeping students engaged during presentations. Defining precisely how students assess the work of others requires careful consideration. Asking the student for numeric grades is the most common method. However, students tend to assign high grades to most projects. Aggregating peer assessments, therefore, results in all projects receiving the same grade. Moreover, students might strategically assign low grades to the projects of others so that their projects will shine. Asking students to order all projects from best to worst imposes a high cognitive load on them, as studies have shown that people find it difficult to order more than a handful of items. To address these issues, we propose a novel peer rating model, R2R, consisting of (a) an algorithm that elicits student assessments and (b) a protocol for aggregating grades to produce a single order. The algorithm asks students to evaluate projects and answer pairwise comparison queries. These are then aggregated into a ranking over the projects. $R2R$ was deployed and tested in a university course and showed promising results, including fewer ties between alternatives and a significant reduction in the communication load on students.

翻译：在高等教育课程中，同行评估活动是保持学生在演示环节参与度的常见方法。如何精确界定学生评价他人成果的方式需要审慎考量。要求学生给出数值分数是最常用的方法，但学生倾向于给大多数项目打出高分。因此，汇总同行评估结果会导致所有项目得到相同分数。此外，学生可能策略性地给他人项目打低分，以使自己的项目脱颖而出。要求学生将所有项目从优到劣排序会带来较高的认知负荷，研究表明人们难以对超过数个条目进行排序。为解决这些问题，我们提出一种新型同行评级模型R2R，包含：(a) 引导学生评估的算法，以及(b) 汇总评分以生成单一排序的协议。该算法要求学生评估项目并回答成对比较查询，这些查询结果随后被聚合为项目间的排名。R2R已在大学课程中部署测试并取得显著成效，包括减少选项间的并列情况，以及显著降低学生的沟通负荷。