Due to its complexity, graph learning-based multi-modal integration and classification is one of the most challenging obstacles for disease prediction. To effectively offset the negative impact between modalities in the process of multi-modal integration and extract heterogeneous information from graphs, we propose a novel method called MMKGL (Multi-modal Multi-Kernel Graph Learning). For the problem of negative impact between modalities, we propose a multi-modal graph embedding module to construct a multi-modal graph. Different from conventional methods that manually construct static graphs for all modalities, each modality generates a separate graph by adaptive learning, where a function graph and a supervision graph are introduced for optimization during the multi-graph fusion embedding process. We then propose a multi-kernel graph learning module to extract heterogeneous information from the multi-modal graph. The information in the multi-modal graph at different levels is aggregated by convolutional kernels with different receptive field sizes, followed by generating a cross-kernel discovery tensor for disease prediction. Our method is evaluated on the benchmark Autism Brain Imaging Data Exchange (ABIDE) dataset and outperforms the state-of-the-art methods. In addition, discriminative brain regions associated with autism are identified by our model, providing guidance for the study of autism pathology.

翻译：由于其复杂性，基于图学习的多模态融合与分类是疾病预测中最具挑战性的障碍之一。为有效抵消多模态融合过程中模态间的负面影响并提取图中的异质信息，我们提出了一种名为MMKGL（多模态多核图学习）的新方法。针对模态间的负向影响问题，我们提出了多模态图嵌入模块以构建多模态图。与传统方法为所有模态手动构建静态图不同，每种模态通过自适应学习生成独立图，其中在多图融合嵌入过程中引入函数图与监督图进行优化。进一步提出多核图学习模块以提取多模态图中的异质信息。通过不同感受野大小的卷积核聚合多模态图中不同层级的信息，进而生成跨核发现张量用于疾病预测。我们的方法在基准数据集自闭症脑成像数据交换（ABIDE）上进行了评估，并优于现有最优方法。此外，该模型识别出与自闭症相关的判别性脑区，为自闭症病理研究提供了指导。