In this paper, we consider an experimental setting where units enter the experiment sequentially. Our goal is to form stopping rules which lead to estimators of treatment effects with a given precision. We propose a fixed-width confidence interval design (FWCID) where the experiment terminates once a pre-specified confidence interval width is achieved. We show that under this design, the difference-in-means estimator is a consistent estimator of the average treatment effect and standard confidence intervals have asymptotic guarantees of coverage and efficiency for several versions of the design. In addition, we propose a version of the design that we call fixed power design (FPD) where a given power is asymptotically guaranteed for a given treatment effect, without the need to specify the variances of the outcomes under treatment or control. In addition, this design also gives a consistent difference-in-means estimator with correct coverage of the corresponding standard confidence interval. We complement our theoretical findings with Monte Carlo simulations where we compare our proposed designs with standard designs in the sequential experiments literature, showing that our designs outperform these designs in several important aspects. We believe our results to be relevant for many experimental settings where units enter sequentially, such as in clinical trials, as well as in online A/B tests used by the tech and e-commerce industry.

翻译：本文考虑实验单元序贯进入的实验场景，旨在构造能够以给定精度估计处理效应的停止规则。我们提出了一种固定宽度置信区间设计（FWCID），当达到预设的置信区间宽度时实验即终止。我们证明，在该设计下，均值差分估计量是平均处理效应的一致性估计量，标准置信区间在设计的多个版本中具有渐近覆盖率和效率保证。此外，我们提出一种变体设计——固定功效设计（FPD），它在给定处理效应下能渐近保证特定功效，且无需指定处理组或对照组结果的方差。该设计同样能给出均值差分估计量的一致性估计，并保证对应标准置信区间的正确覆盖率。我们通过蒙特卡洛模拟补充理论结果，将所提设计与序贯实验文献中的标准设计进行比较，结果表明我们的设计在多个重要方面优于现有方案。我们认为本文结论适用于单元序贯进入的多种实验场景，例如临床试验，以及科技与电商行业使用的在线A/B测试。