Understanding the impact of treatment effect over time is a fundamental aspect of many scientific and medical studies. In this paper, we introduce a novel approach under a continuous-time reinforcement learning framework for testing a treatment effect. Specifically, our method provides an effective test on carryover effects of treatment over time utilizing the average treatment effect (ATE). The average treatment effect is defined as difference of value functions over an infinite horizon, which accounts for cumulative treatment effects, both immediate and carryover. The proposed method outperforms existing testing procedures such as discrete time reinforcement learning strategies in multi-resolution observation settings where observation times can be irregular. Another advantage of the proposed method is that it can capture treatment effects of a shorter duration and provide greater accuracy compared to discrete-time approximations, through the use of continuous-time estimation for the value function. We establish the asymptotic normality of the proposed test statistics and apply it to OhioT1DM diabetes data to evaluate the cumulative treatment effects of bolus insulin on patients' glucose levels.

翻译：理解治疗效果随时间的影响是众多科学与医学研究的基本课题。本文在连续时间强化学习框架下，提出了一种检验治疗效果的新方法。具体而言，本方法利用平均处理效应（ATE）对治疗随时间产生的遗留效应进行有效检验。平均处理效应定义为无限时间跨度上价值函数的差值，该定义同时考虑了即时治疗效果与遗留效应构成的累积治疗效果。在观测时间可能不规律的多分辨率观测场景中，所提方法的表现优于离散时间强化学习策略等现有检验流程。本方法的另一优势在于，通过对价值函数进行连续时间估计，能够捕捉持续时间更短的治疗效果，且相比离散时间近似方法具有更高的精度。我们建立了所提检验统计量的渐近正态性，并将其应用于OhioT1DM糖尿病数据集，以评估推注胰岛素对患者血糖水平的累积治疗效果。