Conditioning on variables affected by treatment can induce post-treatment bias when estimating causal effects. Although this suggests that researchers should measure potential moderators before administering the treatment in an experiment, doing so may also bias causal effect estimation if the covariate measurement primes respondents to react differently to the treatment. This paper formally analyzes this trade-off between post-treatment and priming biases in three experimental designs that vary when moderators are measured: pre-treatment, post-treatment, or a randomized choice between the two. We derive nonparametric bounds for interactions between the treatment and the moderator in each design and show how to use substantive assumptions to narrow these bounds. These bounds allow researchers to assess the sensitivity of their empirical findings to either source of bias. We extend the basic framework in two ways. First, we apply the framework to the case of post-treatment attention checks and bound how much inattentive respondents can attenuate estimated treatment effects. Second, we develop a parametric Bayesian approach to incorporate pre-treatment covariates in the analysis to sharpen our inferences and quantify estimation uncertainty. We apply these methods to a survey experiment on electoral messaging. We conclude with practical recommendations for scholars designing experiments.
翻译:在估计因果效应时,以受处理影响后的变量为条件可能诱发事后处理偏差。尽管这意味着研究者应在实验处理前测量潜在调节变量,但若协变量测量使受试者对处理产生差异化反应,同样可能造成因果效应估计偏差。本文系统分析了事后处理偏差与启动偏差在三种实验设计(处理前测量、处理后测量、随机选择测量时机)中的权衡关系。我们推导了每种设计中处理与调节变量交互作用的非参数边界,并展示了如何利用实质性假设收窄这些边界。这些边界使研究者能够评估实证结果对两类偏差的敏感程度。我们从两个维度拓展了基础分析框架:其一,将该框架应用于处理后注意力核查场景,界定了不专注受试者可能削弱处理效应估计值的上限;其二,开发了参数化贝叶斯方法以整合处理前协变量,从而优化推断并量化估计不确定性。我们通过选举信息调查实验验证了这些方法,最终为设计实验的研究者提出实践建议。