A/B testing is a widely-used paradigm within marketing optimization because it promises identification of causal effects and because it is implemented out of the box in most messaging delivery software platforms. Modern businesses, however, often run many A/B/n tests at the same time and in parallel, and package many content variations into the same messages, not all of which are part of an explicit test. Whether as the result of many teams testing at the same time, or as part of a more sophisticated reinforcement learning (RL) approach that continuously adapts tests and test condition assignment based on previous results, dynamic parallel testing cannot be evaluated the same way traditional A/B tests are evaluated. This paper presents a method for disentangling the causal effects of the various tests under conditions of continuous test adaptation, using a matched-synthetic control group that adapts alongside the tests.

翻译：A/B测试是营销优化中广泛使用的范式，因其能够识别因果效应，且多数消息投递软件平台均具备开箱即用的实现功能。然而，现代企业常同时并行开展多项A/B/n测试，并在同一条消息中集成多种内容变体，其中并非所有变体都属于明确的测试范畴。无论是多团队同步测试的结果，还是基于先前结果持续调整测试及测试条件分配的更复杂的强化学习方法的组成部分，动态并行测试都无法沿用传统A/B测试的评估方式进行评估。本文提出了一种方法，可在持续测试调整条件下，利用随测试同步调整的匹配合成对照组，厘清各类测试的因果效应。