We study the design and analysis of switchback experiments conducted on a single aggregate unit. The design problem is to partition the continuous time space into intervals and switch treatments between intervals, in order to minimize the estimation error of the treatment effect. We show that the estimation error depends on four factors: carryover effects, periodicity, serially correlated outcomes, and impacts from simultaneous experiments. We derive a rigorous bias-variance decomposition and show the tradeoffs of the estimation error from these factors. The decomposition provides three new insights in choosing a design: First, balancing the periodicity between treated and control intervals reduces the variance; second, switching less frequently reduces the bias from carryover effects while increasing the variance from correlated outcomes, and vice versa; third, randomizing interval start and end points reduces both bias and variance from simultaneous experiments. Combining these insights, we propose a new empirical Bayes design approach. This approach uses prior data and experiments for designing future experiments. We illustrate this approach using real data from a ride-sharing platform, yielding a design that reduces MSE by 33% compared to the status quo design used on the platform.

翻译：我们研究在单一聚合单元上进行的切换实验的设计与分析。设计问题在于将连续时间空间划分为区间，并在区间之间切换处理，以最小化处理效应的估计误差。我们证明估计误差取决于四个因素：遗留效应、周期性、序列相关结果以及来自同步实验的影响。我们推导出严格的偏差-方差分解，并展示了这些因素导致的估计误差权衡。该分解为选择设计提供了三个新见解：首先，平衡处理区间与控制区间之间的周期性可降低方差；其次，降低切换频率可减少遗留效应带来的偏差，但会增加相关结果导致的方差，反之亦然；第三，随机化区间起始点和结束点可降低同步实验带来的偏差和方差。结合这些见解，我们提出了一种新的经验贝叶斯设计方法。该方法利用先验数据和实验来设计未来实验。我们通过网约车平台的实际数据展示了该方法，所得设计相比平台当前使用的设计将均方误差降低了33%。