Large-scale online platforms and marketplace systems often evaluate new policies through experiments that randomize treatment across operational units (e.g., geographies, regions, or clusters) over many time periods. In these settings, standard A/B testing can be inefficient or unreliable due to a limited number of units, substantial cross-unit heterogeneity, non-stationarity, and potential carryover across periods. We propose Sequentially-Rerandomized Switchback Experiments (SRSB), a new experimental design that helps mitigate these challenges. SRSB re-randomizes treatment at each time period such as to enforce balance on pre-specified prognostic variables constructed from past observations. In the absence of carryover, SRSB improves precision by leveraging temporal dependence through balancing lagged outcomes and covariates; we develop finite-sample randomization inference under a sharp null as well as asymptotic inference as the number of periods grows. We then extend SRSB to settings with first-order carryover and introduce a blocked SRSB variant that rerandomizes within strata defined by the previous treatment to form stable and comparable "stay" groups. Extensive simulations demonstrate the practical gains and robustness of SRSB relative to standard switchback designs.
翻译:大规模在线平台和市场化系统常通过跨多个时间段、对运营单元(如地理区域、地区或集群)进行随机分组的实验来评估新策略。在此类场景中,标准A/B测试因单元数量有限、单元间异质性显著、时间非平稳性以及潜在的跨期干扰效应,可能导致效率低下或结果不可靠。本文提出序列重随机开关实验(Sequentially-Rerandomized Switchback Experiments,简称SRSB),一种有助于缓解上述挑战的新型实验设计方法。SRSB在每个时间段重新随机分配处理方案,以在基于历史观测数据预先指定的预后变量上实现均衡。在无跨期干扰的情况下,SRSB通过平衡滞后期结果与协变量来利用时间依赖性,从而提升精度;我们发展了在尖锐零假设下的有限样本随机化推断方法,以及随时间周期数增长而渐近的推断理论。随后将SRSB扩展至具有一阶跨期干扰的场景,并引入分块SRSB变体,该变体在先前期处理定义的层内进行重随机,以形成稳定且可比的"滞留"组。大规模仿真实验表明,与标准开关设计相比,SRSB具有显著的实践增益和鲁棒性。