Large datasets often contain multiple distinct feature sets, or views, that offer complementary information that can be exploited by multi-view learning methods to improve results. We investigate anatomical multi-view data, where each brain anatomical structure is described with multiple feature sets. In particular, we focus on sets of white matter microstructure and connectivity features from diffusion MRI, as well as sets of gray matter area and thickness features from structural MRI. We investigate machine learning methodology that applies multi-view approaches to improve the prediction of non-imaging phenotypes, including demographics (age), motor (strength), and cognition (picture vocabulary). We present an explainable multi-view network (EMV-Net) that can use different anatomical views to improve prediction performance. In this network, each individual anatomical view is processed by a view-specific feature extractor and the extracted information from each view is fused using a learnable weight. This is followed by a wavelet transform-based module to obtain complementary information across views which is then applied to calibrate the view-specific information. Additionally, the calibrator produces an attention-based calibration score to indicate anatomical structures' importance for interpretation.

翻译：大型数据集通常包含多个不同的特征集（即视图），这些视图提供互补信息，可被多视图学习方法利用以提升结果。本研究探究解剖多视图数据，其中每个脑解剖结构由多个特征集描述。具体而言，我们重点研究扩散磁共振成像中的白质微结构和连接性特征集，以及结构磁共振成像中的灰质面积和厚度特征集。我们研究应用多视图方法改善非影像表型（包括人口统计学特征（年龄）、运动能力（力量）和认知能力（图片词汇））预测的机器学习方法。提出一种可解释多视图网络（EMV-Net），该网络可利用不同解剖视图提升预测性能。在该网络中，每个解剖视图由特定视图的特征提取器处理，并利用可学习权重融合各视图提取的信息。随后通过基于小波变换的模块获取跨视图的互补信息，并以此校准视图特定信息。此外，校准器生成基于注意力的校准分数，用于指示解剖结构对解释的重要性。