Brain network analysis is a useful approach to studying human brain disorders because it can distinguish patients from healthy people by detecting abnormal connections. Due to the complementary information from multiple modal neuroimages, multimodal fusion technology has a lot of potential for improving prediction performance. However, effective fusion of multimodal medical images to achieve complementarity is still a challenging problem. In this paper, a novel hierarchical structural-functional connectivity fusing (HSCF) model is proposed to construct brain structural-functional connectivity matrices and predict abnormal brain connections based on functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI). Specifically, the prior knowledge is incorporated into the separators for disentangling each modality of information by the graph convolutional networks (GCN). And a disentangled cosine distance loss is devised to ensure the disentanglement's effectiveness. Moreover, the hierarchical representation fusion module is designed to effectively maximize the combination of relevant and effective features between modalities, which makes the generated structural-functional connectivity more robust and discriminative in the cognitive disease analysis. Results from a wide range of tests performed on the public Alzheimer's Disease Neuroimaging Initiative (ADNI) database show that the proposed model performs better than competing approaches in terms of classification evaluation. In general, the proposed HSCF model is a promising model for generating brain structural-functional connectivities and identifying abnormal brain connections as cognitive disease progresses.

翻译：脑网络分析是研究人类脑部疾病的有效方法，因为它能通过检测异常连接来区分患者与健康人群。由于多模态神经影像具有互补信息，多模态融合技术在提升预测性能方面潜力巨大。然而，有效融合多模态医学图像以实现互补性仍是一个具有挑战性的问题。本文提出了一种新颖的层次化结构-功能连接融合（HSCF）模型，用于构建脑结构-功能连接矩阵，并基于功能磁共振成像（fMRI）和扩散张量成像（DTI）预测异常脑连接。具体而言，该模型将先验知识融入分离器中，通过图卷积网络（GCN）解耦每种模态的信息，并设计了解耦余弦距离损失函数以确保解耦的有效性。此外，层次化表示融合模块被设计用于有效最大化模态间相关且有效特征的组合，使得生成的结构-功能连接在认知疾病分析中更具鲁棒性和判别性。在公共阿尔茨海默病神经影像学倡议（ADNI）数据库上进行的大量实验结果表明，所提模型在分类评估上优于现有对比方法。总体而言，所提出的HSCF模型是一种有前景的模型，可用于生成脑结构-功能连接，并识别随认知疾病进展的异常脑连接。