In online platforms, the impact of a treatment on an observed outcome may change over time as 1) users learn about the intervention, and 2) the system personalization, such as individualized recommendations, change over time. We introduce a non-parametric causal model of user actions in a personalized system. We show that the Cookie-Cookie-Day (CCD) experiment, designed for the measurement of the user learning effect, is biased when there is personalization. We derive new experimental designs that intervene in the personalization system to generate the variation necessary to separately identify the causal effect mediated through user learning and personalization. Making parametric assumptions allows for the estimation of long-term causal effects based on medium-term experiments. In simulations, we show that our new designs successfully recover the dynamic causal effects of interest.

翻译：在在线平台中，处理对观测结果的影响可能随时间发生变化，原因有二：1) 用户逐渐了解干预措施，2) 系统个性化（如个性化推荐）随时间演变。我们提出一个在个性化系统中用户行为的非参数因果模型。研究表明，用于测量用户学习效应的"Cookie-Cookie-Day"（CCD）实验在存在个性化时会产生偏差。我们推导出新的实验设计，通过干预个性化系统来产生必要的变异，从而单独识别通过用户学习和个性化所中介的因果效应。引入参数假设使我们能够基于中期实验估计长期因果效应。仿真结果表明，我们的新设计成功恢复了感兴趣的动态因果效应。