In multi-armed bandit (MAB) experiments, it is often advantageous to continuously produce inference on the average treatment effect (ATE) between arms as new data arrive and determine a data-driven stopping time for the experiment. We develop the Mixture Adaptive Design (MAD), a new experimental design for multi-armed bandit experiments that produces powerful and anytime-valid inference on the ATE for \emph{any} bandit algorithm of the experimenter's choice, even those without probabilistic treatment assignment. Intuitively, the MAD "mixes" any bandit algorithm of the experimenter's choice with a Bernoulli design through a tuning parameter $\delta_t$, where $\delta_t$ is a deterministic sequence that decreases the priority placed on the Bernoulli design as the sample size grows. We prove that for $\delta_t = \omega\left(t^{-1/4}\right)$, the MAD generates anytime-valid asymptotic confidence sequences that are guaranteed to shrink around the true ATE. Hence, the experimenter is guaranteed to detect a true non-zero treatment effect in finite time. Additionally, we prove that the regret of the MAD approaches that of its underlying bandit algorithm over time, and hence, incurs a relatively small loss in regret in return for powerful inferential guarantees. Finally, we conduct an extensive simulation study exhibiting that the MAD achieves finite-sample anytime validity and high power without significant losses in finite-sample reward.

翻译：在多臂老虎机（MAB）实验中，随着新数据的持续抵达，通常需要连续推断各臂之间的平均处理效应（ATE），并确定基于数据的实验停止时间。我们提出了混合自适应设计（MAD）——一种面向多臂老虎机实验的新型实验设计方法，该方法能够对研究者选择的任意老虎机算法（即便是未采用概率化处理分配的算法）所产生的ATE，提供强效且任意有效的推断。直观而言，MAD通过一个调谐参数$\delta_t$将研究者选定的任意老虎机算法与伯努利设计进行"混合"，其中$\delta_t$是一个确定性序列，随着样本量的增加而降低对伯努利设计的优先级。我们证明：当$\delta_t = \omega\left(t^{-1/4}\right)$时，MAD可生成任意有效的渐近置信序列，且该序列必定收敛至真实ATE附近。因此，研究者保证能在有限时间内检测出真实非零处理效应。此外，我们还证明MAD的累积遗憾值随时间趋近于其底层老虎机算法的遗憾值，即仅以较小的遗憾损失换取强大的推断保证。最后，通过广泛仿真研究证明：MAD在未显著损失有限样本奖励的前提下，实现了有限样本的任意有效性与高统计效力。