We propose an integrated deep-generative framework, that jointly models complementary information from resting-state functional MRI (rs-fMRI) connectivity and diffusion tensor imaging (DTI) tractography to extract predictive biomarkers of a disease. The generative part of our framework is a structurally-regularized Dynamic Dictionary Learning (sr-DDL) model that decomposes the dynamic rs-fMRI correlation matrices into a collection of shared basis networks and time varying patient-specific loadings. This matrix factorization is guided by the DTI tractography matrices to learn anatomically informed connectivity profiles. The deep part of our framework is an LSTM-ANN block, which models the temporal evolution of the patient sr-DDL loadings to predict multidimensional clinical severity. Our coupled optimization procedure collectively estimates the basis networks, the patient-specific dynamic loadings, and the neural network weights. We validate our framework on a multi-score prediction task in 57 patients diagnosed with Autism Spectrum Disorder (ASD). Our hybrid model outperforms state-of-the-art baselines in a five-fold cross validated setting and extracts interpretable multimodal neural signatures of brain dysfunction in ASD.

翻译：我们提出了一种集成的深度-生成框架，该框架联合建模来自静息态功能磁共振成像（rs-fMRI）连接性与扩散张量成像（DTI）纤维束成像的互补信息，以提取疾病的预测性生物标志物。我们框架的生成部分是一个结构正则化动态字典学习（sr-DDL）模型，该模型将动态rs-fMRI相关矩阵分解为一组共享的基础网络和随时间变化的患者特定载荷。该矩阵分解过程由DTI纤维束成像矩阵引导，以学习解剖学信息化的连接性特征。我们框架的深度部分是一个LSTM-ANN模块，它对患者sr-DDL载荷的时间演化进行建模，以预测多维临床严重程度。我们的耦合优化过程共同估计了基础网络、患者特异性动态载荷以及神经网络权重。我们在57名被诊断为自闭症谱系障碍（ASD）的患者身上验证了我们的框架，用于一项多评分预测任务。我们的混合模型在五折交叉验证设置中优于最先进的基线方法，并提取了ASD中脑功能障碍的可解释多模态神经特征。