A/B testing is a common approach used in industry to facilitate innovation through the introduction of new features or the modification of existing software. Traditionally, A/B tests are conducted sequentially, with each experiment targeting the entire population of the corresponding application. This approach can be time-consuming and costly, particularly when the experiments are not relevant to the entire population. To tackle these problems, we introduce a new self-adaptive approach called AutoPABS, short for Automated Pipelines of A/B tests using Self-adaptation, that (1) automates the execution of pipelines of A/B tests, and (2) supports a split of the population in the pipeline to divide the population into multiple A/B tests according to user-based criteria, leveraging machine learning. We started the evaluation with a small survey to probe the appraisal of the notation and infrastructure of AutoPABS. Then we performed a series of tests to measure the gains obtained by applying a population split in an automated A/B testing pipeline, using an extension of the SEAByTE artifact. The survey results show that the participants express the usefulness of automating A/B testing pipelines and population split. The tests show that automatically executing pipelines of A/B tests with a population split accelerates the identification of statistically significant results of the parallel executed experiments of A/B tests compared to a traditional approach that performs the experiments sequentially.

翻译：A/B测试是工业界广泛采用的一种方法，通过引入新功能或修改现有软件来促进创新。传统上，A/B测试按顺序进行，每个实验面向整个应用的用户群体。这种方法可能耗时且成本高昂，尤其当实验与全体用户不相关时。为解决这些问题，我们提出了一种名为AutoPABS（Automated Pipelines of A/B tests using Self-adaptation）的新型自适应方法，该方法能够（1）自动化执行A/B测试管道，（2）支持在管道中进行人群分割，利用机器学习将用户群根据用户特征划分为多个A/B测试。我们首先通过一项小规模调查来评估AutoPABS符号体系和基础设施的有效性，随后基于SEAByTE工具的扩展，测量了在自动化A/B测试管道中应用人群分割所获得的效率提升。调查结果表明，参与者认可自动化A/B测试管道与人群分割的实用性。实验显示，相较于传统顺序执行方式，采用人群分割的自动化A/B测试管道能够加速并行实验中统计显著结果的识别。