We consider the performance of the difference-in-means estimator in a two-arm randomized experiment under common experimental endpoints such as continuous (regression), incidence, proportion and survival. We examine performance under both equal and unequal allocation to treatment groups and we consider both the Neyman randomization model and the population model. We show that in the Neyman model, where the only source of randomness is the treatment manipulation, there is no free lunch: complete randomization is minimax for the estimator's mean squared error. In the population model, where each subject experiences response noise with zero mean, the optimal design is the deterministic perfect-balance allocation. However, this allocation is generally NP-hard to compute and moreover, depends on unknown response parameters. When considering the tail criterion of Kapelner et al. (2021), we show the optimal design is less random than complete randomization and more random than the deterministic perfect-balance allocation. We prove that Fisher's blocking design provides the asymptotically optimal degree of experimental randomness. Theoretical results are supported by simulations in all considered experimental settings.

翻译：摘要：本文研究了两臂随机实验中，在连续（回归）、发生率、比例和生存等常见实验终点下，均值差估计量的表现。我们分析了治疗组在等分配与不等分配下的性能，并同时考虑了内曼随机化模型和总体模型。在内曼模型中，唯一随机性来源为治疗分配，我们证明了“没有免费午餐”：完全随机化对于估计量的均方误差而言是极小极大最优的。在总体模型中，每个受试者经历零均值响应噪声，最优设计是确定性完美平衡分配。然而，这种分配通常计算困难（NP难问题），且依赖于未知的响应参数。针对Kapelner等人（2021）提出的尾部准则，我们证明了最优设计比完全随机化更少随机性，而比确定性完美平衡分配更多随机性。我们进一步证明Fisher分组设计提供了实验随机性的渐近最优程度。所有实验结果均通过仿真在各类实验场景中得到验证。