Early identification of stroke is crucial for intervention, requiring reliable models. We proposed an efficient retinal image representation together with clinical information to capture a comprehensive overview of cardiovascular health, leveraging large multimodal datasets for new medical insights. Our approach is one of the first contrastive frameworks that integrates graph and tabular data, using vessel graphs derived from retinal images for efficient representation. This method, combined with multimodal contrastive learning, significantly enhances stroke prediction accuracy by integrating data from multiple sources and using contrastive learning for transfer learning. The self-supervised learning techniques employed allow the model to learn effectively from unlabeled data, reducing the dependency on large annotated datasets. Our framework showed an AUROC improvement of 3.78% from supervised to self-supervised approaches. Additionally, the graph-level representation approach achieved superior performance to image encoders while significantly reducing pre-training and fine-tuning runtimes. These findings indicate that retinal images are a cost-effective method for improving cardiovascular disease predictions and pave the way for future research into retinal and cerebral vessel connections and the use of graph-based retinal vessel representations.

翻译：卒中的早期识别对于干预至关重要，这需要可靠的模型。我们提出了一种结合临床信息的高效视网膜图像表征方法，以捕捉心血管健康的全面概览，并利用大规模多模态数据集获取新的医学洞见。我们的方法是首批整合图数据与表格数据的对比学习框架之一，利用从视网膜图像提取的血管图进行高效表征。该方法与多模态对比学习相结合，通过整合多源数据并利用对比学习进行迁移学习，显著提升了卒中预测的准确性。所采用的自监督学习技术使模型能够从未标注数据中有效学习，降低了对大规模标注数据集的依赖。我们的框架显示，从监督方法到自监督方法的AUROC提升了3.78%。此外，图级表征方法在显著减少预训练和微调运行时间的同时，取得了优于图像编码器的性能。这些发现表明，视网膜图像是改善心血管疾病预测的一种经济有效的方法，并为未来研究视网膜与脑血管连接以及基于图的视网膜血管表征的应用铺平了道路。