In many business applications, including online marketing and customer churn prevention, randomized controlled trials (RCT's) are conducted to investigate on the effect of specific treatment (coupon offers, advertisement mailings,...). Such RCT's allow for the estimation of average treatment effects as well as the training of (uplift) models for the heterogeneity of treatment effects between individuals. The problem with these RCT's is that they are costly and this cost increases with the number of individuals included into the RCT. For this reason, there is research how to conduct experiments involving a small number of individuals while still obtaining precise treatment effect estimates. We contribute to this literature a heteroskedasticity-aware stratified sampling (HS) scheme, which leverages the fact that different individuals have different noise levels in their outcome and precise treatment effect estimation requires more observations from the "high-noise" individuals than from the "low-noise" individuals. By theory as well as by empirical experiments, we demonstrate that our HS-sampling yields significantly more precise estimates of the ATE, improves uplift models and makes their evaluation more reliable compared to RCT data sampled completely randomly. Due to the relative ease of application and the significant benefits, we expect HS-sampling to be valuable in many real-world applications.

翻译：在许多商业应用中，包括在线营销和客户流失预防，随机对照试验（RCT）被用于研究特定处理（优惠券、广告邮件等）的效果。这类随机对照试验既可用于估计平均处理效应，也能训练用于评估个体间处理效应异质性的（增量）模型。这些随机对照试验的问题在于成本高昂，且成本随纳入试验的个体数量增加而上升。因此，已有研究探讨如何通过包含少量个体的实验仍能获得精确的处理效应估计。本文提出一种异方差感知的分层抽样（HS）方案，利用不同个体结果变量噪声水平存在差异的特性：精确的处理效应估计需要从“高噪声”个体中比“低噪声”个体获取更多观测数据。通过理论推导和实证实验，我们证明：与完全随机抽样的随机对照试验数据相比，我们的HS抽样能显著提升平均处理效应（ATE）的估计精度，改善增量模型，并使其评估更加可靠。由于应用相对简便且效益显著，我们预期HS抽样将在众多实际应用中发挥重要价值。