Pretraining has proven to be a powerful technique in natural language processing (NLP), exhibiting remarkable success in various NLP downstream tasks. However, in the medical domain, existing pretrained models on electronic health records (EHR) fail to capture the hierarchical nature of EHR data, limiting their generalization capability across diverse downstream tasks using a single pretrained model. To tackle this challenge, this paper introduces a novel, general, and unified pretraining framework called MEDHMP, specifically designed for hierarchically multimodal EHR data. The effectiveness of the proposed MEDHMP is demonstrated through experimental results on eight downstream tasks spanning three levels. Comparisons against eighteen baselines further highlight the efficacy of our approach.

翻译：预训练已被证明是自然语言处理（NLP）中的一项强大技术，在各种NLP下游任务中展现出显著的成功。然而，在医疗领域，现有的基于电子健康记录（EHR）的预训练模型未能捕捉EHR数据的分层特性，限制了其在单一预训练模型下跨不同下游任务的泛化能力。为应对这一挑战，本文提出了一种新颖、通用且统一的预训练框架MEDHMP，专门设计用于处理分层多模态的EHR数据。通过在覆盖三个层面的八项下游任务上的实验结果，展示了所提出的MEDHMP的有效性。与十八个基线的对比进一步凸显了本方法的优越性。