In recent years, large amounts of electronic health records (EHRs) concerning chronic diseases, such as cancer, diabetes, and mental disease, have been collected to facilitate medical diagnosis. Modeling the dynamic properties of EHRs related to chronic diseases can be efficiently done using dynamic treatment regimes (DTRs), which are a set of sequential decision rules. While Reinforcement learning (RL) is a widely used method for creating DTRs, there is ongoing research in developing RL algorithms that can effectively handle large amounts of data. In this paper, we present a novel approach, a distributed Q-learning algorithm, for generating DTRs. The novelties of our research are as follows: 1) From a methodological perspective, we present a novel and scalable approach for generating DTRs by combining distributed learning with Q-learning. The proposed approach is specifically designed to handle large amounts of data and effectively generate DTRs. 2) From a theoretical standpoint, we provide generalization error bounds for the proposed distributed Q-learning algorithm, which are derived within the framework of statistical learning theory. These bounds quantify the relationships between sample size, prediction accuracy, and computational burden, providing insights into the performance of the algorithm. 3) From an applied perspective, we demonstrate the effectiveness of our proposed distributed Q-learning algorithm for DTRs by applying it to clinical cancer treatments. The results show that our algorithm outperforms both traditional linear Q-learning and commonly used deep Q-learning in terms of both prediction accuracy and computation cost.

翻译：近年来，针对癌症、糖尿病和精神疾病等慢性疾病，已收集大量电子健康记录（EHRs）以辅助医学诊断。利用动态治疗方案（DTRs）——即一组连续决策规则——可有效建模慢性疾病EHRs的动态特性。尽管强化学习（RL）是构建DTRs的常用方法，但开发能高效处理海量数据的RL算法仍是当前研究热点。本文提出一种新型分布式Q学习算法以生成DTRs，其创新性体现在：1）方法论层面，通过融合分布式学习与Q学习，提出一种可扩展的新型DTRs生成方法，该方法专为处理大规模数据而设计；2）理论层面，在统计学习理论框架下推导了所提分布式Q学习算法的泛化误差界，量化了样本量、预测精度与计算负担之间的关联，揭示了算法性能的深层机理；3）应用层面，通过将其应用于临床癌症治疗验证了该分布式Q学习算法在DTRs中的有效性。结果表明，本算法在预测精度与计算成本两方面均优于传统线性Q学习及常用深度Q学习。