Online user-generated content platforms allocate billions of dollars of promotional traffic through algorithms in two-sided marketplaces. To evaluate updates to these algorithms, platforms frequently rely on creator-side randomized experiments. However, because treated and control creators compete for exposure, such experiments suffer from algorithmic interference: exposure outcomes depend on competitors' treatment status. We show that commonly used difference-in-means estimators can therefore be severely biased and may even recommend deploying inferior algorithms. To address this challenge, we develop a structured semiparametric framework that explicitly models the competitive allocation mechanism underlying exposure. Our approach combines an algorithm choice model that characterizes how exposure is allocated across competing content with a viewer response model that captures engagement conditional on exposure. We construct a debiased estimator grounded in the double machine learning framework to recover the global treatment effect of platform-wide rollout. Methodologically, we extend DML asymptotic theory to accommodate correlated samples arising from overlapping consideration sets. Using Monte Carlo simulations and a large-scale field experiment on a major short-video platform, we show that our estimator closely matches an interference-free benchmark obtained from a costly double-sided experimental design. In contrast, standard estimators exhibit substantial bias and, in some cases, even reverse the sign of the effect.
翻译:在线用户生成内容平台通过双边市场中的算法分配数十亿美元的推广流量。为评估这些算法的更新,平台常依赖创作者侧的随机实验。然而,由于处理组和对照组的创作者会争夺曝光机会,此类实验受到算法干预的影响:曝光结果取决于竞争对手的处理状态。我们证明,常用的均值差分估计量因此可能存在严重偏差,甚至可能建议部署劣质算法。为应对这一挑战,我们建立了一个结构化半参数框架,显式建模曝光背后的竞争性分配机制。该方法结合了刻画竞争内容间曝光分配机制的算法选择模型,以及捕捉曝光条件下参与度的观众响应模型。我们基于双重机器学习框架构建了去偏估计量,以恢复平台全局部署的整体处理效应。在方法论上,我们扩展了DML渐近理论以容纳由重叠考虑集产生的相关样本。通过蒙特卡洛模拟和某大型短视频平台的大规模现场实验,我们证明该估计量与通过成本高昂的双侧实验设计获得的无干预基准高度吻合。相比之下,标准估计量表现出显著偏差,在某些情况下甚至逆转了效应方向。