Mediation analysis learns the causal effect transmitted via mediator variables between treatments and outcomes and receives increasing attention in various scientific domains to elucidate causal relations. Most existing works focus on point-exposure studies where each subject only receives one treatment at a single time point. However, there are a number of applications (e.g., mobile health) where the treatments are sequentially assigned over time and the dynamic mediation effects are of primary interest. Proposing a reinforcement learning (RL) framework, we are the first to evaluate dynamic mediation effects in settings with infinite horizons. We decompose the average treatment effect into an immediate direct effect, an immediate mediation effect, a delayed direct effect, and a delayed mediation effect. Upon the identification of each effect component, we further develop robust and semi-parametrically efficient estimators under the RL framework to infer these causal effects. The superior performance of the proposed method is demonstrated through extensive numerical studies, theoretical results, and an analysis of a mobile health dataset.

翻译：中介分析用于学习通过中介变量在治疗与结局之间传递的因果效应，并因阐明因果关系的需求在多个科学领域受到日益关注。现有研究大多聚焦于单次暴露研究，其中每个受试者仅在单个时间点接受一次治疗。然而，在诸多应用场景（例如移动健康）中，治疗是随时间顺序分配的，动态中介效应成为主要关注点。我们首次提出一个强化学习（RL）框架，用于评估无限时间视野下的动态中介效应。我们将平均处理效应分解为即时直接效应、即时中介效应、延迟直接效应和延迟中介效应。在识别出每个效应成分后，我们进一步在RL框架下开发了鲁棒且半参数有效的估计器来推断这些因果效应。通过大量数值研究、理论结果以及对移动健康数据集的分析，我们证明了所提方法的优越性能。