Mediation analysis learns the causal effect transmitted via mediator variables between treatments and outcomes and receives increasing attention in various scientific domains to elucidate causal relations. Most existing works focus on point-exposure studies where each subject only receives one treatment at a single time point. However, there are a number of applications (e.g., mobile health) where the treatments are sequentially assigned over time and the dynamic mediation effects are of primary interest. Proposing a reinforcement learning (RL) framework, we are the first to evaluate dynamic mediation effects in settings with infinite horizons. We decompose the average treatment effect into an immediate direct effect, an immediate mediation effect, a delayed direct effect, and a delayed mediation effect. Upon the identification of each effect component, we further develop robust and semi-parametrically efficient estimators under the RL framework to infer these causal effects. The superior performance of the proposed method is demonstrated through extensive numerical studies, theoretical results, and an analysis of a mobile health dataset.

翻译：中介分析旨在学习通过中介变量在治疗与结果之间传递的因果效应，并在多个科学领域中日益受到关注以阐明因果关系。现有研究大多聚焦于单次暴露研究，即每个受试者仅在单个时间点接受一次治疗。然而，在许多应用场景（例如移动健康）中，治疗会随时间顺序分配，且动态中介效应是主要关注点。通过提出一种强化学习框架，我们首次在无限时间范围的情境下评估了动态中介效应。我们将平均处理效应分解为即时直接效应、即时中介效应、延迟直接效应和延迟中介效应。在识别出各效应成分后，我们进一步在强化学习框架下开发了稳健且半参数有效的估计量，用于推断这些因果效应。通过广泛的数值研究、理论结果以及对一个移动健康数据集的分析，验证了所提出方法的优越性能。