Comparative judgement studies elicit quality assessments through pairwise comparisons, typically analysed using the Bradley-Terry model. A challenge in these studies is experimental design, specifically, determining the optimal pairs to compare to maximize statistical efficiency. Constructing static experimental designs for these studies requires spectral decomposition of a covariance matrix over pairs of pairs, which becomes computationally infeasible for studies with more than approximately 150 objects. We propose a scalable method based on reduced basis decomposition that bypasses explicit construction of this matrix, achieving computational savings of two to three orders of magnitude. We establish eigenvalue bounds guaranteeing approximation quality and characterise the rank structure of the design matrix. Simulations demonstrate speedup factors exceeding 100 for studies with 64 or more objects, with negligible approximation error. We apply the method to construct designs for a 452-region spatial study in under 7 minutes and enable real-time design updates for classroom peer assessment, reducing computation time from 15 minutes to 15 seconds.

翻译：比较判断研究通过成对比较获取质量评估，通常采用Bradley-Terry模型进行分析。此类研究的关键挑战在于实验设计，特别是如何确定最优比较对以最大化统计效率。构建静态实验设计需要计算成对组合协方差矩阵的谱分解，当研究对象超过约150个时，该计算将变得不可行。我们提出一种基于降基分解的可扩展方法，通过规避显式构建该矩阵，实现计算效率提升两到三个数量级。我们建立了保证近似质量的特征值边界，并刻画了设计矩阵的秩结构特征。仿真实验表明，对于64个及以上对象的研究，加速因子超过100倍且近似误差可忽略不计。我们将该方法应用于构建包含452个区域的空间研究设计，耗时不足7分钟；在课堂同伴评估中实现了实时设计更新，将计算时间从15分钟缩短至15秒。