Gaining insights into the structural and functional mechanisms of the brain has been a longstanding focus in neuroscience research, particularly in the context of understanding and treating neuropsychiatric disorders such as Schizophrenia (SZ). Nevertheless, most of the traditional multimodal deep learning approaches fail to fully leverage the complementary characteristics of structural and functional connectomics data to enhance diagnostic performance. To address this issue, we proposed ConneX, a multimodal fusion method that integrates cross-attention mechanism and multilayer perceptron (MLP)-Mixer for refined feature fusion. Modality-specific backbone graph neural networks (GNNs) were firstly employed to obtain feature representation for each modality. A unified cross-modal attention network was then introduced to fuse these embeddings by capturing intra- and inter-modal interactions, while MLP-Mixer layers refined global and local features, leveraging higher-order dependencies for end-to-end classification with a multi-head joint loss. Extensive evaluations demonstrated improved performance on two distinct clinical datasets, highlighting the robustness of our proposed framework.

翻译：解析大脑的结构与功能机制一直是神经科学研究的重要焦点，特别是在理解和治疗如精神分裂症（SZ）等神经精神障碍的背景下。然而，大多数传统的多模态深度学习方法未能充分利用结构连接组学与功能连接组学数据的互补特性以提升诊断性能。为解决这一问题，我们提出了ConneX，一种融合跨注意力机制与多层感知机（MLP）-Mixer进行精细化特征融合的多模态融合方法。首先采用模态特定的主干图神经网络（GNNs）获取各模态的特征表示；随后引入统一的跨模态注意力网络，通过捕获模态内与模态间交互来融合这些嵌入表示，同时利用MLP-Mixer层优化全局与局部特征，借助高阶依赖性进行端到端分类，并采用多头联合损失函数。大量评估结果表明，该方法在两个不同的临床数据集上均实现了性能提升，凸显了所提出框架的鲁棒性。