Digital technology organizations routinely use online experiments (e.g. A/B tests) to guide their product and business decisions. In e-commerce, we often measure changes to transaction- or item-based business metrics such as Average Basket Value (ABV), Average Basket Size (ABS), and Average Selling Price (ASP); yet it remains a common pitfall to ignore the dependency between the value/size of transactions/items during experiment design and analysis. We present empirical evidence on such dependency, its impact on measurement uncertainty, and practical implications on A/B test outcomes if left unmitigated. By making the evidence available, we hope to drive awareness of the pitfall among experimenters in e-commerce and hence encourage the adoption of established mitigation approaches. We also share lessons learned when incorporating selected mitigation approaches into our experimentation analysis platform currently in production.

翻译：数字技术组织通常使用在线实验（例如A/B测试）来指导产品和业务决策。在电商领域，我们经常衡量基于交易或商品的业务指标变化，例如平均购物篮价值（ABV）、平均购物篮大小（ABS）和平均销售价格（ASP）。然而，在实验设计和分析中忽视交易/商品价值或大小之间的依赖性仍是一个常见陷阱。我们提供了关于这种依赖性、其对测量不确定性的影响以及若不加以缓解对A/B测试结果实际影响的实证证据。通过公开这些证据，我们希望提高电商实验人员对此陷阱的认知，从而推动采用已建立的缓解方法。我们还分享了将选定的缓解方法整合到当前生产中的实验分析平台时获得的经验教训。