Electronic Health Records (EHR) data contains medical records such as diagnoses, medications, procedures, and treatments of patients. This data is often considered sensitive medical information. Therefore, the EHR data from the medical centers often cannot be shared, making it difficult to create prediction models using multi-center EHR data, which is essential for such models' robustness and generalizability. Federated Learning (FL) is an algorithmic approach that allows learning a shared model using data in multiple locations without the need to store all data in a central place. An example of a prediction model's task is to predict future diseases. More specifically, the model needs to predict patient's next visit diagnoses, based on current and previous clinical data. Such a prediction model can support care providers in making clinical decisions and even provide preventive treatment. We propose a federated learning approach for learning medical concepts embedding. This pre-trained model can be used for fine-tuning for specific downstream tasks. Our approach is based on an embedding model like BEHRT, a deep neural sequence transduction model for EHR. We train using federated learning, both the Masked Language Modeling (MLM) and the next visit downstream model. We demonstrate our approach on the MIMIC-IV dataset. We compare the performance of a model trained with FL against a model trained on centralized data. We find that our federated learning approach reaches very close to the performance of a centralized model, and it outperforms local models in terms of average precision. We also show that pre-trained MLM improves the model's average precision performance in the next visit prediction task, compared to an MLM model without pre-training. Our code is available at https://github.com/nadavlab/FederatedBEHRT.

翻译：电子健康记录（EHR）数据包含患者的诊断、用药、手术及治疗等医疗记录，这类数据通常被视为敏感医疗信息。因此，各医疗中心的EHR数据往往无法共享，这使得利用多中心EHR数据构建预测模型（对模型鲁棒性和泛化能力至关重要）变得困难。联邦学习（FL）是一种算法框架，允许在无需集中存储所有数据的情况下，利用多地的数据学习共享模型。预测模型的任务之一是预测未来疾病，具体而言，模型需根据当前及既往临床数据预测患者下次就诊的诊断信息。这类预测模型可辅助医疗人员制定临床决策，甚至提供预防性治疗。本文提出一种用于学习医学概念嵌入的联邦学习方法，该预训练模型可针对特定下游任务进行微调。我们的方法基于嵌入模型（如BEHRT），这是一种用于EHR的深度神经序列转导模型。我们利用联邦学习同时训练掩码语言建模（MLM）和下次就诊下游模型。在MIMIC-IV数据集上验证方法后，我们比较了联邦训练模型与集中训练模型的性能。结果发现：联邦学习方法的性能非常接近集中训练模型，且平均精度优于本地模型。此外，与未预训练的MLM模型相比，预训练MLM在下次就诊预测任务中能显著提升模型平均精度性能。相关代码已开源至 https://github.com/nadavlab/FederatedBEHRT。