Renal transplantation emerges as the most effective solution for end-stage renal disease. Occurring from complex causes, a substantial risk of transplant chronic dysfunction persists and may lead to graft loss. Medical imaging plays a substantial role in renal transplant monitoring in clinical practice. However, graft supervision is multi-disciplinary, notably joining nephrology, urology, and radiology, while identifying robust biomarkers from such high-dimensional and complex data for prognosis is challenging. In this work, taking inspiration from the recent success of Large Language Models (LLMs), we propose MEDIMP -- Medical Images with clinical Prompts -- a model to learn meaningful multi-modal representations of renal transplant Dynamic Contrast-Enhanced Magnetic Resonance Imaging (DCE MRI) by incorporating structural clinicobiological data after translating them into text prompts. MEDIMP is based on contrastive learning from joint text-image paired embeddings to perform this challenging task. Moreover, we propose a framework that generates medical prompts using automatic textual data augmentations from LLMs. Our goal is to learn meaningful manifolds of renal transplant DCE MRI, interesting for the prognosis of the transplant or patient status (2, 3, and 4 years after the transplant), fully exploiting the limited available multi-modal data most efficiently. Extensive experiments and comparisons with other renal transplant representation learning methods with limited data prove the effectiveness of MEDIMP in a relevant clinical setting, giving new directions toward medical prompts. Our code is available at https://github.com/leomlck/MEDIMP.

翻译：肾移植是终末期肾病最有效的治疗方案。由于病因复杂，慢性移植功能障碍的风险持续存在，并可能导致移植物丢失。医学影像在临床上对肾移植监测起着重要作用。然而，移植物监控涉及多学科协作，尤其是肾病学、泌尿学和放射学，从这类高维复杂数据中识别稳健的生物标志物用于预后极具挑战性。受大规模语言模型（LLM）近期成功的启发，本研究提出MEDIMP——临床提示下的医学影像——一个通过学习肾移植动态对比增强磁共振成像（DCE MRI）多模态表征的模型，该模型将结构化临床生物学数据转换为文本提示后融入学习。MEDIMP基于联合文本-图像配对嵌入的对比学习来执行这一艰巨任务。此外，我们提出一个框架，利用LLM自动生成文本数据增强来构建医学提示。我们的目标是学习肾移植DCE MRI有意义的流形表征，这些表征对预测移植或患者状态（移植后第2、3和4年）具有重要价值，同时最有效地利用有限的多模态数据。与现有肾移植表征学习方法在有限数据条件下的广泛实验和比较表明，MEDIMP在相关临床环境中具有有效性，为医学提示研究开辟了新方向。我们的代码开源地址：https://github.com/leomlck/MEDIMP。