Dynamic treatment regimes (DTRs) formalize medical decision-making as a sequence of rules for different stages, mapping patient-level information to recommended treatments. In practice, estimating an optimal DTR using observational data from electronic medical record (EMR) databases can be complicated by nonignorable missing covariates resulting from informative monitoring of patients. Since complete case analysis can provide consistent estimation of outcome model parameters under the assumption of outcome-independent missingness, Q-learning is a natural approach to accommodating nonignorable missing covariates. However, the backward induction algorithm used in Q-learning can introduce challenges, as nonignorable missing covariates at later stages can result in nonignorable missing pseudo-outcomes at earlier stages, leading to suboptimal DTRs, even if the longitudinal outcome variables are fully observed. To address this unique missing data problem in DTR settings, we propose two weighted Q-learning approaches where inverse probability weights for missingness of the pseudo-outcomes are obtained through estimating equations with valid nonresponse instrumental variables or sensitivity analysis. The asymptotic properties of the weighted Q-learning estimators are derived, and the finite-sample performance of the proposed methods is evaluated and compared with alternative methods through extensive simulation studies. Using EMR data from the Medical Information Mart for Intensive Care database, we apply the proposed methods to investigate the optimal fluid strategy for sepsis patients in intensive care units.

翻译：动态治疗方案（DTRs）将医疗决策形式化为不同阶段的规则序列，将患者层面的信息映射到推荐的治疗方案。在实践中，使用来自电子病历（EMR）数据库的观察数据估计最优DTR可能因患者信息性监测导致的不可忽略缺失协变量而变得复杂。由于在结果独立缺失假设下，完整病例分析可以提供结果模型参数的一致性估计，Q学习是处理不可忽略缺失协变量的自然方法。然而，Q学习中使用的后向归纳算法可能带来挑战，因为后期阶段的不可忽略缺失协变量可能导致早期阶段出现不可忽略的伪结果缺失，从而产生次优DTR，即使纵向结果变量被完全观测。为解决DTR设置中这一独特的缺失数据问题，我们提出了两种加权Q学习方法，其中通过具有有效无应答工具变量的估计方程或敏感性分析获得伪结果缺失的逆概率权重。推导了加权Q学习估计量的渐近性质，并通过大量模拟研究评估和比较了所提方法的有限样本性能与替代方法。利用来自重症监护医疗信息集市数据库的EMR数据，我们应用所提方法研究了重症监护室脓毒症患者的最佳液体策略。