Alongside neuroimaging such as MRI scans and PET, Alzheimer's disease (AD) datasets contain valuable tabular data including AD biomarkers and clinical assessments. Existing computer vision approaches struggle to utilize this additional information. To address these needs, we propose a generalizable framework for multimodal contrastive learning of image data and tabular data, a novel tabular attention module for amplifying and ranking salient features in tables, and the application of these techniques onto Alzheimer's disease prediction. Experimental evaulations demonstrate the strength of our framework by detecting Alzheimer's disease (AD) from over 882 MR image slices from the ADNI database. We take advantage of the high interpretability of tabular data and our novel tabular attention approach and through attribution of the attention scores for each row of the table, we note and rank the most predominant features. Results show that the model is capable of an accuracy of over 83.8%, almost a 10% increase from previous state of the art.

翻译：除了核磁共振成像（MRI）和正电子发射断层扫描（PET）等神经影像学手段外，阿尔茨海默病（AD）数据集还包含有价值的表格数据，包括AD生物标志物和临床评估。现有的计算机视觉方法难以充分利用这些附加信息。为应对这些需求，我们提出了一种适用于图像数据与表格数据多模态对比学习的通用框架，一种用于增强并排序表格中显著特征的新型表格注意力模块，并将这些技术应用于阿尔茨海默病预测。实验评估表明，通过从ADNI数据库中的882张以上MR图像切片检测阿尔茨海默病（AD），我们的框架展现出强大性能。我们利用了表格数据的高可解释性以及新型表格注意力方法，通过归因表格每行的注意力分数，记录并排序了最显著的特征。结果显示，该模型的准确率可达83.8%以上，相较先前最先进水平提升了近10%。