Two-sided platforms are central to modern commerce and content sharing and often utilize A/B testing for developing new features. While user-side experiments are common, seller-side experiments become crucial for specific interventions and metrics. This paper investigates the effects of interference caused by feedback loops on seller-side experiments in two-sided platforms, with a particular focus on the counterfactual interleaving design, proposed in \citet{ha2020counterfactual,nandy2021b}. These feedback loops, often generated by pacing algorithms, cause outcomes from earlier sessions to influence subsequent ones. This paper contributes by creating a mathematical framework to analyze this interference, theoretically estimating its impact, and conducting empirical evaluations of the counterfactual interleaving design in real-world scenarios. Our research shows that feedback loops can result in misleading conclusions about the treatment effects.

翻译：双面平台是现代商业与内容共享的核心，常采用A/B测试来开发新功能。虽然用户侧实验较为常见，但针对特定干预措施和指标，卖方侧实验变得至关重要。本文研究反馈回路引发的干扰对双面平台卖方侧实验的影响，特别关注文献\citet{ha2020counterfactual,nandy2021b}提出的反事实交叠设计。这些反馈回路通常由节奏算法生成，导致早期会话的结果影响后续会话。本文通过构建数学框架分析此类干扰，从理论角度估计其影响，并在实际场景中对反事实交叠设计进行实证评估。研究表明，反馈回路可能导致对处理效应的误导性结论。