Clinical diagnosis guidelines aim at specifying the steps that may lead to a diagnosis. Guidelines enable rationalizing and normalizing clinical decisions but suffer drawbacks as they are built to cover the majority of the population and may fail in guiding to the right diagnosis for patients with uncommon conditions or multiple pathologies. Moreover, their updates are long and expensive, making them unsuitable to emerging practices. Inspired by guidelines, we formulate the task of diagnosis as a sequential decision-making problem and study the use of Deep Reinforcement Learning (DRL) algorithms trained on Electronic Health Records (EHRs) to learn the optimal sequence of observations to perform in order to obtain a correct diagnosis. Because of the variety of DRL algorithms and of their sensitivity to the context, we considered several approaches and settings that we compared to each other, and to classical classifiers. We experimented on a synthetic but realistic dataset to differentially diagnose anemia and its subtypes and particularly evaluated the robustness of various approaches to noise and missing data as those are frequent in EHRs. Within the DRL algorithms, Dueling DQN with Prioritized Experience Replay, and Dueling Double DQN with Prioritized Experience Replay show the best and most stable performances. In the presence of imperfect data, the DRL algorithms show competitive, but less stable performances when compared to the classifiers (Random Forest and XGBoost); although they enable the progressive generation of a pathway to the suggested diagnosis, which can both guide or explain the decision process.

翻译：临床诊断指南旨在明确可能导致诊断的步骤。指南有助于规范临床决策，但其缺点是针对多数人群设计，对于患有罕见病症或多重病理的患者可能无法引导至正确诊断。此外，指南更新周期长且成本高昂，因此难以适应新兴实践。受指南启发，我们将诊断任务构建为序列决策问题，并研究使用基于电子健康记录（EHR）训练的深度强化学习（DRL）算法来学习获取正确诊断所需的最优观察序列。考虑到DRL算法的多样性及其对上下文的敏感性，我们比较了多种方法与设置，并将其与传统分类器进行对比。我们在合成但逼真的数据集上实验了贫血及其亚型的鉴别诊断，特别评估了各类方法对噪声和缺失数据的鲁棒性——这些在EHR中极为常见。在DRL算法中，采用优先经验回放的Dueling DQN和采用优先经验回放的Dueling Double DQN表现出最佳且最稳定的性能。在数据不完善的情况下，DRL算法虽展现出竞争性但稳定性弱于分类器（随机森林和XGBoost）；不过它们能逐步生成诊断路径，既可指导也可解释决策过程。