Renal transplantation emerges as the most effective solution for end-stage renal disease. Occurring from complex causes, a substantial risk of transplant chronic dysfunction persists and may lead to graft loss. Medical imaging plays a substantial role in renal transplant monitoring in clinical practice. However, graft supervision is multi-disciplinary, notably joining nephrology, urology, and radiology, while identifying robust biomarkers from such high-dimensional and complex data for prognosis is challenging. In this work, taking inspiration from the recent success of Large Language Models (LLMs), we propose MEDIMP -- Medical Images and Prompts -- a model to learn meaningful multi-modal representations of renal transplant Dynamic Contrast-Enhanced Magnetic Resonance Imaging (DCE MRI) by incorporating structural clinicobiological data after translating them into text prompts. MEDIMP is based on contrastive learning from joint text-image paired embeddings to perform this challenging task. Moreover, we propose a framework that generates medical prompts using automatic textual data augmentations from LLMs. Our goal is to learn meaningful manifolds of renal transplant DCE MRI, interesting for the prognosis of the transplant or patient status (2, 3, and 4 years after the transplant), fully exploiting the available multi-modal data in the most efficient way. Extensive experiments and comparisons with other renal transplant representation learning methods with limited data prove the effectiveness of MEDIMP in a relevant clinical setting, giving new directions toward medical prompts. Our code is available at https://github.com/leomlck/MEDIMP.

翻译：中文摘要：肾移植是终末期肾病最有效的治疗方案。然而，由于病因复杂，移植后慢性功能障碍的风险持续存在，可能导致移植物丢失。临床实践中，医学影像在肾移植监测中发挥着重要作用。但移植物监护涉及多学科协作，特别是肾病学、泌尿外科和放射学，且从这些高维复杂数据中识别稳健的预后生物标志物极具挑战性。受大型语言模型（LLMs）近期成功应用的启发，本文提出MEDIMP（医学影像与提示）模型，通过将结构性临床生物学数据转化为文本提示，实现肾移植动态对比增强磁共振成像（DCE MRI）的多模态表征学习。该模型基于联合文本-图像配对嵌入的对比学习来执行这一具有挑战性的任务。此外，我们提出一个框架，利用LLMs自动生成医学文本数据增强。本研究旨在学习肾移植DCE MRI的有意义流形结构，为移植体或患者状态（术后2、3、4年）的预后提供依据，并以最高效的方式充分利用现有模态数据。与有限数据条件下其他肾移植表征学习方法的广泛实验和比较证明，MEDIMP在相关临床场景中具有显著效果，为医学提示的发展提供了新方向。相关代码已开源：https://github.com/leomlck/MEDIMP。