Online A/B testing is widely used in the internet industry to inform decisions on new feature roll-outs. For online marketplaces (such as advertising markets), standard approaches to A/B testing may lead to biased results when buyers operate under a budget constraint, as budget consumption in one arm of the experiment impacts performance of the other arm. To counteract this interference, one can use a budget-split design where the budget constraint operates on a per-arm basis and each arm receives an equal fraction of the budget, leading to ``budget-controlled A/B testing.'' Despite clear advantages of budget-controlled A/B testing, performance degrades when budget are split too small, limiting the overall throughput of such systems. In this paper, we propose a parallel budget-controlled A/B testing design where we use market segmentation to identify submarkets in the larger market, and we run parallel experiments on each submarket. Our contributions are as follows: First, we introduce and demonstrate the effectiveness of the parallel budget-controlled A/B test design with submarkets in a large online marketplace environment. Second, we formally define market interference in first-price auction markets using the first price pacing equilibrium (FPPE) framework. Third, we propose a debiased surrogate that eliminates the first-order bias of FPPE, drawing upon the principles of sensitivity analysis in mathematical programs. Fourth, we derive a plug-in estimator for the surrogate and establish its asymptotic normality. Fifth, we provide an estimation procedure for submarket parallel budget-controlled A/B tests. Finally, we present numerical examples on semi-synthetic data, confirming that the debiasing technique achieves the desired coverage properties.

翻译：在线A/B测试广泛应用于互联网行业，以指导新功能上线的决策。对于在线市场（如广告市场），当买方在预算约束下运作时，标准的A/B测试方法可能导致结果偏差，因为实验某一分支的预算消耗会影响另一分支的性能。为消除此干扰，可采用预算分割设计，使预算约束按分支独立运行，每个分支获得等额预算，从而形成“预算控制型A/B测试”。尽管预算控制型A/B测试优势明显，但预算分割过细会降低性能，限制此类系统的整体吞吐量。本文提出一种并行预算控制型A/B测试设计，通过市场细分识别大型市场中的子市场，并在各子市场并行开展实验。具体贡献如下：首先，我们在大型在线市场环境中引入并验证了基于子市场的并行预算控制型A/B测试设计的有效性；其次，利用一阶定价均衡（FPPE）框架正式定义了一阶拍卖市场中的市场干扰；再次，借鉴数学规划敏感性分析原理，提出一种消除FPPE一阶偏差的去偏替代量；第四，推导了该替代量的插件估计量并证明其渐近正态性；第五，提出子市场并行预算控制型A/B测试的估计流程；最后，通过半合成数据数值实验验证了去偏技术实现所需覆盖性质的能力。