Incrementality experiments compare customers exposed to a marketing action designed to increase sales to those randomly assigned to a control group. These experiments suffer from noisy responses which make precise estimation of the average treatment effect (ATE) and marketing ROI difficult. We develop a model that improves the precision by estimating separate treatment effects for three latent strata defined by potential outcomes in the experiment -- customers who would buy regardless of ad exposure, those who would buy only if exposed to ads and those who would not buy regardless. The overall ATE is estimated by averaging the strata-level effects, and this produces a more precise estimator of the ATE over a wide range of conditions typical of marketing experiments. Analytical results and simulations show that the method decreases the sampling variance of the ATE most when (1) there are large differences in the treatment effect between latent strata and (2) the model used to estimate the strata-level effects is well-identified. Applying the procedure to 5 catalog experiments shows a reduction of 30-60% in the variance of the overall ATE. This leads to a substantial decrease in decision errors when the estimator is used to determine whether ads should be continued or discontinued.

翻译：增量实验通过比较接触旨在提升销量的营销行为的客户与随机分配至对照组的客户，以评估营销效果。此类实验常受噪声响应干扰，导致平均处理效应（ATE）及营销投资回报率的精确估计困难。我们提出一种模型，通过基于实验中的潜在结果划分三个潜在层次（无论是否接触广告均会购买的客户、仅接触广告后购买的客户、无论是否接触广告均不购买的客户），分别估计各层次的处理效应，从而提升估计精度。整体ATE通过加权各层次效应计算，在典型营销实验的广泛条件下能生成更精确的ATE估计量。理论分析与仿真结果表明，当（1）潜在层次间处理效应差异显著，且（2）用于估计层次效应的模型可识别性良好时，该方法能最大程度降低ATE的抽样方差。将本方法应用于5个目录实验后，整体ATE方差降低30-60%。当使用该估计量判断广告应继续或停止投放时，该方法显著减少了决策失误。