We consider the task of evaluating policies of algorithmic resource allocation through randomized controlled trials (RCTs). Such policies are tasked with optimizing the utilization of limited intervention resources, with the goal of maximizing the benefits derived. Evaluation of such allocation policies through RCTs proves difficult, notwithstanding the scale of the trial, because the individuals' outcomes are inextricably interlinked through resource constraints controlling the policy decisions. Our key contribution is to present a new estimator leveraging our proposed novel concept, that involves retrospective reshuffling of participants across experimental arms at the end of an RCT. We identify conditions under which such reassignments are permissible and can be leveraged to construct counterfactual trials, whose outcomes can be accurately ascertained, for free. We prove theoretically that such an estimator is more accurate than common estimators based on sample means -- we show that it returns an unbiased estimate and simultaneously reduces variance. We demonstrate the value of our approach through empirical experiments on synthetic, semi-synthetic as well as real case study data and show improved estimation accuracy across the board.

翻译：我们考虑通过随机对照试验（RCT）评估算法资源分配策略的任务。此类策略旨在优化有限干预资源的利用，以最大化所带来的效益。通过RCT评估此类分配策略十分困难，无论试验规模如何，因为个体的结果与制约策略决策的资源约束密不可分地相互关联。我们的核心贡献是提出一种新估计量，该估计量利用我们提出的新颖概念，即在RCT结束时对跨实验组的参与者进行回顾性重排。我们确定了此类重排允许的条件，并说明如何利用这些条件构建反事实试验，且能免费准确获取其结果。我们从理论上证明，该估计量比基于样本均值的常见估计量更精确——我们表明它能返回无偏估计并同时降低方差。我们通过在合成数据、半合成数据以及真实案例研究数据上的实证实验展示了该方法的价值，并在所有场景中均观察到估计精度的提升。