In this paper, we examine the biases arising in A/B tests where a firm modifies a continuous parameter, such as price, to estimate the global treatment effect associated to a given performance metric. Such biases emerge from canonical designs and estimators due to interference among market participants. We employ structural modeling and differential calculus to derive intuitive structural characterizations of this bias. We then specialize our general model to a standard revenue management pricing problem. This setting highlights a key potential pitfall in the use of pricing experiments to guide profit maximization: notably, the canonical estimator for the change in profits can have the {\em wrong sign}. In other words, following the guidance of the canonical estimator may lead the firm to move prices in the wrong direction, and thereby decrease profits relative to the status quo. We apply these results to a two-sided market model and show how this ``change of sign" regime depends on model parameters, and discuss structural and practical implications for platform operators.
翻译:本文研究企业通过A/B测试调整价格等连续参数时,在估计特定绩效指标的全局处理效应中产生的偏差。此类偏差源于市场参与者之间的干扰效应,是经典实验设计与估计方法的固有产物。我们采用结构建模与微分演算方法,推导出该偏差的直观结构化表征。进而将通用模型特化为标准收益管理定价问题,该场景揭示了价格实验指导利润最大化中的关键潜在陷阱:值得注意的是,利润变化量的规范估计量可能出现符号错误。换言之,遵循规范估计量的指导可能导致企业向错误方向调整价格,从而相对于现状降低利润。我们将这些结论应用于双边市场模型,论证这种"符号反转"机制如何依赖于模型参数,并探讨其对平台运营者的结构与实践启示。