The ability to predict the future trajectory of a patient is a key step toward the development of therapeutics for complex diseases such as Alzheimer's disease (AD). However, most machine learning approaches developed for prediction of disease progression are either single-task or single-modality models, which can not be directly adopted to our setting involving multi-task learning with high dimensional images. Moreover, most of those approaches are trained on a single dataset (i.e. cohort), which can not be generalized to other cohorts. We propose a novel multimodal multi-task deep learning model to predict AD progression by analyzing longitudinal clinical and neuroimaging data from multiple cohorts. Our proposed model integrates high dimensional MRI features from a 3D convolutional neural network with other data modalities, including clinical and demographic information, to predict the future trajectory of patients. Our model employs an adversarial loss to alleviate the study-specific imaging bias, in particular the inter-study domain shifts. In addition, a Sharpness-Aware Minimization (SAM) optimization technique is applied to further improve model generalization. The proposed model is trained and tested on various datasets in order to evaluate and validate the results. Our results showed that 1) our model yields significant improvement over the baseline models, and 2) models using extracted neuroimaging features from 3D convolutional neural network outperform the same models when applied to MRI-derived volumetric features.

翻译：预测患者未来疾病轨迹是开发阿尔茨海默病（AD）等复杂疾病治疗方法的关键步骤。然而，大多数用于疾病进展预测的机器学习方法要么是单任务模型，要么是单模态模型，无法直接应用于涉及高维图像的多任务学习场景。此外，这些方法大多在单一数据集（即队列）上训练，难以泛化到其他队列。我们提出了一种新颖的多模态多任务深度学习模型，通过分析来自多个队列的纵向临床和神经影像数据来预测AD进展。该模型将三维卷积神经网络提取的高维MRI特征与临床信息、人口统计学等其他数据模态相结合，以预测患者的未来轨迹。模型采用对抗性损失来减轻研究特异性影像偏差，特别是研究间域偏移。同时，应用锐度感知最小化（SAM）优化技术进一步改善模型泛化能力。所提出的模型在多个数据集上进行了训练和测试以评估验证结果。结果显示：1）该模型相比基线模型取得了显著提升；2）使用三维卷积神经网络提取的神经影像特征的模型，其表现优于应用MRI衍生体积特征的同类型模型。