In this paper, we study the design and analysis of experiments conducted on a set of units over multiple time periods where the starting time of the treatment may vary by unit. The design problem involves selecting an initial treatment time for each unit in order to most precisely estimate both the instantaneous and cumulative effects of the treatment. We first consider non-adaptive experiments, where all treatment assignment decisions are made prior to the start of the experiment. For this case, we show that the optimization problem is generally NP-hard, and we propose a near-optimal solution. Under this solution, the fraction entering treatment each period is initially low, then high, and finally low again. Next, we study an adaptive experimental design problem, where both the decision to continue the experiment and treatment assignment decisions are updated after each period's data is collected. For the adaptive case, we propose a new algorithm, the Precision-Guided Adaptive Experiment (PGAE) algorithm, that addresses the challenges at both the design stage and at the stage of estimating treatment effects, ensuring valid post-experiment inference accounting for the adaptive nature of the design. Using realistic settings, we demonstrate that our proposed solutions can reduce the opportunity cost of the experiments by over 50%, compared to static design benchmarks.
翻译:本文研究了在多个时间段内对一组单元进行实验的设计与分析问题,其中处理组的启动时间可能因单元而异。该设计问题涉及为每个单元选择初始处理时间,以最精确地估计处理的即时效应和累积效应。我们首先考虑非自适应性实验,即所有处理分配决策均在实验开始前完成。针对这种情况,我们证明优化问题通常是NP难的,并提出了一种近最优解。在该解下,每个时期进入处理组的比例最初较低,随后升高,最后再次降低。接着,我们研究自适应性实验设计问题,其中继续实验的决策和处理分配决策均会在每个时期数据收集后更新。针对自适应情况,我们提出了一种新算法——精度引导的自适应实验(PGAE)算法,该算法解决了设计阶段和估计处理效应阶段的挑战,并确保了考虑自适应设计性质的有效实验后推断。通过现实场景的验证,我们证明了所提方案相比静态设计基准可将实验的机会成本降低超过50%。