We present a new text-to-SQL dataset for electronic health records (EHRs). The utterances were collected from 222 hospital staff members, including physicians, nurses, and insurance review and health records teams. To construct the QA dataset on structured EHR data, we conducted a poll at a university hospital and used the responses to create seed questions. We then manually linked these questions to two open-source EHR databases, MIMIC-III and eICU, and included various time expressions and held-out unanswerable questions in the dataset, which were also collected from the poll. Our dataset poses a unique set of challenges: the model needs to 1) generate SQL queries that reflect a wide range of needs in the hospital, including simple retrieval and complex operations such as calculating survival rate, 2) understand various time expressions to answer time-sensitive questions in healthcare, and 3) distinguish whether a given question is answerable or unanswerable. We believe our dataset, EHRSQL, can serve as a practical benchmark for developing and assessing QA models on structured EHR data and take a step further towards bridging the gap between text-to-SQL research and its real-life deployment in healthcare. EHRSQL is available at https://github.com/glee4810/EHRSQL.
翻译:我们提出一个新的面向电子健康记录(EHR)的文本到SQL数据集。话语数据来自222名医院工作人员,包括医生、护士以及保险审核和健康记录团队。为了构建结构化EHR数据上的问答数据集,我们在大学医院进行了一项民意调查,并利用调查回复生成种子问题。随后,我们手动将这些问题关联到两个开源EHR数据库(MIMIC-III和eICU),并在数据集中纳入多种时间表达式及从调查中收集的不可回答问题。我们的数据集带来一系列独特挑战:模型需要1)生成反映医院中广泛需求的SQL查询,包括简单检索和计算存活率等复杂操作;2)理解多种时间表达式以回答医疗领域中的时间敏感问题;3)区分给定问题是否可回答。我们相信,本数据集EHRSQL可作为在结构化EHR数据上开发与评估问答模型的实用基准,并进一步弥合文本到SQL研究与其在医疗领域实际部署之间的差距。EHRSQL可通过https://github.com/glee4810/EHRSQL获取。