In the past decade, the technology industry has adopted online randomized controlled experiments (a.k.a. A/B testing) to guide product development and make business decisions. In practice, A/B tests are often implemented with increasing treatment allocation: the new treatment is gradually released to an increasing number of units through a sequence of randomized experiments. In scenarios such as experimenting in a social network setting or in a bipartite online marketplace, interference among units may exist, which can harm the validity of simple inference procedures. In this work, we introduce a widely applicable procedure to test for interference in A/B testing with increasing allocation. Our procedure can be implemented on top of an existing A/B testing platform with a separate flow and does not require a priori a specific interference mechanism. In particular, we introduce two permutation tests that are valid under different assumptions. Firstly, we introduce a general statistical test for interference requiring no additional assumption. Secondly, we introduce a testing procedure that is valid under a time fixed effect assumption. The testing procedure is of very low computational complexity, it is powerful, and it formalizes a heuristic algorithm implemented already in industry. We demonstrate the performance of the proposed testing procedure through simulations on synthetic data. Finally, we discuss one application at LinkedIn, where a screening step is implemented to detect potential interference in all their marketplace experiments with the proposed methods in the paper.

翻译：过去十年间，科技行业已采用在线随机对照实验（即A/B测试）来指导产品开发并制定商业决策。实践中，A/B测试常通过渐扩处理分配实施：新方案通过一系列随机实验逐步释放给越来越多的单元。在社交网络环境或双边在线市场等场景中，单元间可能存在干扰，这会损害简单推断程序的有效性。本研究提出一种广泛适用的程序，用于检测渐扩分配A/B测试中的干扰。该程序可在现有A/B测试平台基础上通过独立流程实现，且无需预先假设特定干扰机制。具体而言，我们引入两种在不同假设下有效的置换检验：首先提出无需额外假设的通用干扰统计检验，其次提出在时间固定效应假设下有效的检验程序。该检验程序计算复杂度极低、检验效能高，并形式化了已在工业界应用的启发式算法。我们通过合成数据模拟验证了所提检验程序的性能，最后讨论了其在领英（LinkedIn）的应用案例——该平台采用本文方法对所有市场实验实施干扰筛查步骤。