Motivation: Disease diagnosis oriented dialogue system models the interactive consultation procedure as Markov Decision Process and reinforcement learning algorithms are used to solve the problem. Existing approaches usually employ a flat policy structure that treat all symptoms and diseases equally for action making. This strategy works well in the simple scenario when the action space is small, however, its efficiency will be challenged in the real environment. Inspired by the offline consultation process, we propose to integrate a hierarchical policy structure of two levels into the dialogue systemfor policy learning. The high-level policy consists of amastermodel that is responsible for triggering a low-levelmodel, the lowlevel policy consists of several symptom checkers and a disease classifier. The proposed policy structure is capable to deal with diagnosis problem including large number of diseases and symptoms. Results: Experimental results on three real-world datasets and a synthetic dataset demonstrate that our hierarchical framework achieves higher accuracy and symptom recall in disease diagnosis compared with existing systems. We construct a benchmark including datasets and implementation of existing algorithms to encourage follow-up researches. Availability: The code and data is available from https://github.com/FudanDISC/DISCOpen-MedBox-DialoDiagnosis Contact: [email protected] Supplementary information: Supplementary data are available at Bioinformatics online.

翻译：动机：面向疾病诊断的对话系统将交互式问诊过程建模为马尔可夫决策过程，并采用强化学习算法进行求解。现有方法通常采用扁平化策略结构，对所有症状和疾病进行同等处理以生成动作。该策略在动作空间较小的简单场景中表现良好，但在真实环境中的效率会受到挑战。受线下问诊流程启发，我们提出将双层分层策略结构集成到对话系统的策略学习中。高层策略包含一个主模型，负责触发低层模型；低层策略由多个症状检查器和一个疾病分类器组成。所提出的策略结构能够处理包含大量疾病和症状的诊断问题。结果：在三个真实世界数据集和一个合成数据集上的实验结果表明，与现有系统相比，我们的分层框架在疾病诊断中实现了更高的准确率和症状召回率。我们构建了一个包含数据集和现有算法实现的基准测试，以鼓励后续研究。可用性：代码和数据可从 https://github.com/FudanDISC/DISCOpen-MedBox-DialoDiagnosis 获取。联系方式：[email protected] 补充信息：补充数据可在 Bioinformatics 在线获取。