Early identification of Mild Cognitive Impairment (MCI) subjects who will eventually progress to Alzheimer Disease (AD) is challenging. Existing deep learning models are mostly single-modality single-task models predicting risk of disease progression at a fixed timepoint. We proposed a multimodal hierarchical multi-task learning approach which can monitor the risk of disease progression at each timepoint of the visit trajectory. Longitudinal visit data from multiple modalities (MRI, cognition, and clinical data) were collected from MCI individuals of the Alzheimer Disease Neuroimaging Initiative (ADNI) dataset. Our hierarchical model predicted at every timepoint a set of neuropsychological composite cognitive function scores as auxiliary tasks and used the forecasted scores at every timepoint to predict the future risk of disease. Relevance weights for each composite function provided explanations about potential factors for disease progression. Our proposed model performed better than state-of-the-art baselines in predicting AD progression risk and the composite scores. Ablation study on the number of modalities demonstrated that imaging and cognition data contributed most towards the outcome. Model explanations at each timepoint can inform clinicians 6 months in advance the potential cognitive function decline that can lead to progression to AD in future. Our model monitored their risk of AD progression every 6 months throughout the visit trajectory of individuals. The hierarchical learning of auxiliary tasks allowed better optimization and allowed longitudinal explanations for the outcome. Our framework is flexible with the number of input modalities and the selection of auxiliary tasks and hence can be generalized to other clinical problems too.

翻译：早期识别轻度认知障碍（MCI）患者中最终会进展为阿尔茨海默病（AD）的个体极具挑战性。现有深度学习模型多为单模态单任务模型，仅能在固定时间点预测疾病进展风险。我们提出一种多模态分层多任务学习方法，该方法可在访视轨迹的每个时间点监测疾病进展风险。研究收集了阿尔茨海默病神经影像学计划（ADNI）数据集中MCI个体的纵向多模态访视数据（MRI、认知评估与临床数据）。我们的分层模型在每个时间点预测一组神经心理综合认知功能评分作为辅助任务，并利用各时间点的预测评分来预测未来疾病风险。各综合功能的关联权重可解释导致疾病进展的潜在因素。我们提出的模型在预测AD进展风险及综合评分方面优于现有最先进基线模型。针对模态数量的消融研究表明，影像与认知数据对结果贡献最大。模型在各时间点的解释信息可提前6个月告知临床医生可能导致未来进展为AD的潜在认知功能衰退。该模型每6个月监测个体在整个访视轨迹中的AD进展风险。辅助任务的分层学习实现了更优的优化效果，并为结果提供了纵向解释。我们的框架在输入模态数量与辅助任务选择方面具有灵活性，因此可推广至其他临床问题。