AI-driven medical history-taking is an important component in symptom checking, automated patient intake, triage, and other AI virtual care applications. As history-taking is extremely varied, machine learning models require a significant amount of data to train. To overcome this challenge, existing systems are developed using indirect data or expert knowledge. This leads to a training-inference gap as models are trained on different kinds of data than what they observe at inference time. In this work, we present a two-stage re-ranking approach that helps close the training-inference gap by re-ranking the first-stage question candidates using a dialogue-contextualized model. For this, we propose a new model, global re-ranker, which cross-encodes the dialogue with all questions simultaneously, and compare it with several existing neural baselines. We test both transformer and S4-based language model backbones. We find that relative to the expert system, the best performance is achieved by our proposed global re-ranker with a transformer backbone, resulting in a 30% higher normalized discount cumulative gain (nDCG) and a 77% higher mean average precision (mAP).

翻译：AI驱动的医疗病史采集是症状检查、自动患者分诊、预检及其他AI虚拟医疗应用中至关重要的组成部分。由于病史采集过程高度多样化，机器学习模型需要大量数据进行训练。为应对这一挑战，现有系统通常采用间接数据或专家知识进行开发，这导致训练与推理阶段存在偏差——模型在训练时所使用的数据类型与推理时实际观测到的数据不一致。本研究提出一种两阶段重排序方法，通过使用对话情境化模型对第一阶段的候选问题重新排序，有效缩小训练-推理差距。为此，我们提出新型全局重排序模型，该模型将对话内容与所有问题同时进行交叉编码，并与多种现有神经基线模型进行对比。实验分别采用基于Transformer和S4的语言模型主干架构。结果表明，相较于专家系统，基于Transformer主干的全局重排序模型取得了最优性能，其归一化折损累积增益（nDCG）提升30%，平均精度均值（mAP）提升77%。