Medical tabular data, abundant in Electronic Health Records (EHRs), is a valuable resource for diverse medical tasks such as risk prediction. While deep learning approaches, particularly transformer-based models, have shown remarkable performance in tabular data prediction, there are still problems remained for existing work to be effectively adapted into medical domain, such as under-utilization of unstructured free-texts, limited exploration of textual information in structured data, and data corruption. To address these issues, we propose P-Transformer, a Prompt-based multimodal Transformer architecture designed specifically for medical tabular data. This framework consists two critical components: a tabular cell embedding generator and a tabular transformer. The former efficiently encodes diverse modalities from both structured and unstructured tabular data into a harmonized language semantic space with the help of pre-trained sentence encoder and medical prompts. The latter integrates cell representations to generate patient embeddings for various medical tasks. In comprehensive experiments on two real-world datasets for three medical tasks, P-Transformer demonstrated the improvements with 10.9%/11.0% on RMSE/MAE, 0.5%/2.2% on RMSE/MAE, and 1.6%/0.8% on BACC/AUROC compared to state-of-the-art (SOTA) baselines in predictability. Notably, the model exhibited strong resilience to data corruption in the structured data, particularly when the corruption rates are high.

翻译：医疗表格数据广泛存在于电子健康记录中，是风险预测等多种医疗任务的重要资源。尽管深度学习方法（尤其是基于Transformer的模型）在表格数据预测中表现显著，但现有工作在有效适应医疗领域时仍存在问题，例如非结构化自由文本利用率不足、结构化数据中文本信息探索有限以及数据损坏。为解决这些问题，我们提出P-Transformer，一种专为医疗表格数据设计的基于提示的多模态Transformer架构。该框架包含两个关键组件：表格单元嵌入生成器和表格Transformer。前者借助预训练句子编码器和医疗提示，将来自结构化和非结构化表格数据的多种模态高效编码为统一的语言语义空间；后者整合单元表示以生成面向多种医疗任务的患者嵌入。在面向三项医疗任务的两个真实数据集上的综合实验中，与最先进基线相比，P-Transformer在预测性能上分别实现了RMSE/MAE提升10.9%/11.0%、0.5%/2.2%以及BACC/AUROC提升1.6%/0.8%。值得注意的是，该模型对结构化数据的数据损坏，尤其是在高损坏率下，表现出强大的鲁棒性。