Brain MRIs are routinely acquired as multiple complementary sequences with unique contrast weighting, including T1-weighed imaging (T1w) anatomic and fluid-sensitive T2-weighted (T2w) contrasts. However, methods for learning unified representations across the multitude of MRI contrast mechanisms at health-system scale are lacking. In this study, we introduce Neuro-JEPA, a sparse multimodal neuroimaging foundation model that combines a latent predictive objective with a Mixture-of-Experts architecture to encode brain MRI across core T1w, T2w, and fluid-suppressed FLAIR imaging (FLAIR). We further provide a systematic methodological study of architectural, masking, objective, and sparsity design choices beneficial for robust neuroimaging multimodal representation learning. Neuro-JEPA was pretrained on 1,551,862 scans from 428,647 studies after modality-specific preprocessing with data curation across three core structural brain MRI sequences. We evaluated the learned representations across clinical and research settings, including 25 tasks from three health systems: NYU Langone, NYU Long Island, and Massachusetts General Hospital, and 22 tasks from 12 public datasets, covering unimodal, multimodal and cross-domain evaluation configurations. Across these benchmarks, existing neuroimaging foundation models showed inconsistent gains over a simple convolutional neural network (CNN) baseline, whereas Neuro-JEPA achieved stronger and more consistent performance across all evaluated settings. These results establish a scalable methodological framework for multimodal neuroimaging representation learning and highlight the need for foundation model evaluation protocols that include simple baselines, clinically heterogeneous cohorts and controlled multimodal comparisons.

翻译：脑部磁共振成像（MRI）通常以多个互补序列的形式常规采集，这些序列具有独特的对比度加权，包括T1加权成像（T1w）解剖对比和液体敏感T2加权成像（T2w）对比。然而，在健康系统规模下，针对多种MRI对比机制学习统一表示的方法尚属空白。本研究提出了Neuro-JEPA，一种稀疏多模态神经影像基础模型，该模型结合了潜在预测目标与混合专家架构，用于对核心T1w、T2w及液体抑制FLAIR成像（FLAIR）序列的脑部MRI进行编码。我们进一步对架构、掩码、目标和稀疏性设计选择进行了系统的方法学研究，这些选择有利于稳健的神经影像多模态表示学习。Neuro-JEPA在完成模态特异性预处理和数据整理后的428,647个研究病例的1,551,862次扫描数据上进行了预训练，覆盖三种核心结构性脑部MRI序列。我们在临床和研究场景下评估了学习到的表示，包括来自三个健康系统（纽约大学朗格尼医学中心、纽约大学长岛医院及麻省总医院）的25个任务，以及来自12个公开数据集的22个任务，涵盖单模态、多模态及跨域评估配置。在这些基准测试中，现有神经影像基础模型相较于简单的卷积神经网络（CNN）基线表现出不一致的性能提升，而Neuro-JEPA在所有评估设置下均实现了更强且更一致的表现。这些成果为多模态神经影像表示学习建立了一个可扩展的方法框架，并强调了在基础模型评估协议中纳入简单基线、临床异质性队列及受控多模态比较的必要性。