Dynamic treatment regimes are sequential decision rules that adapt treatment according to individual time-varying characteristics and outcomes to achieve optimal effects, with applications in precision medicine, personalized recommendations, and dynamic marketing. Estimating optimal dynamic treatment regimes via sequential randomized trials might face costly and ethical hurdles, often necessitating the use of historical observational data. In this work, we utilize proximal causal inference framework for learning optimal dynamic treatment regimes when the unconfoundedness assumption fails. Our contributions are four-fold: (i) we propose three nonparametric identification methods for optimal dynamic treatment regimes; (ii) we establish the semiparametric efficiency bound for the value function of a given regime; (iii) we propose a (K+1)-robust method for learning optimal dynamic treatment regimes, where K is the number of stages; (iv) as a by-product for marginal structural models, we establish identification and estimation of counterfactual means under a static regime. Numerical experiments validate the efficiency and multiple robustness of our proposed methods.

翻译：动态治疗策略是一种根据个体随时间变化的特征和结果调整治疗方案以实现最优效果的序列决策规则，广泛应用于精准医疗、个性化推荐和动态营销等领域。通过序列随机试验估计最优动态治疗策略可能面临成本高昂和伦理障碍，通常需要利用历史观察数据。在本研究中，我们采用近端因果推断框架，在无混杂性假设失效的情况下学习最优动态治疗策略。我们的贡献包括四个方面：(i) 提出了三种最优动态治疗策略的非参数识别方法；(ii) 建立了给定策略价值函数的半参数效率下界；(iii) 提出了一种学习最优动态治疗策略的(K+1)-稳健方法，其中K为治疗阶段数；(iv) 作为边际结构模型的副产品，建立了静态策略下反事实均值的识别与估计方法。数值实验验证了所提方法的效率与多重稳健性。