The prediction of mild cognitive impairment (MCI) conversion to Alzheimer's disease (AD) is important for early treatment to prevent or slow the progression of AD. To accurately predict the MCI conversion to stable MCI or progressive MCI, we propose Triformer, a novel transformer-based framework with three specialized transformers to incorporate multi-model data. Triformer uses I) an image transformer to extract multi-view image features from medical scans, II) a clinical transformer to embed and correlate multi-modal clinical data, and III) a modality fusion transformer that produces an accurate prediction based on fusing the outputs from the image and clinical transformers. Triformer is evaluated on the Alzheimer's Disease Neuroimaging Initiative (ANDI)1 and ADNI2 datasets and outperforms previous state-of-the-art single and multi-modal methods.

翻译：轻度认知障碍（MCI）向阿尔茨海默病（AD）转化的预测对于早期干预以预防或延缓AD进展至关重要。为准确预测MCI转化为稳定型MCI或进展型MCI，我们提出Triformer——一种基于Transformer的新型框架，包含三个专用Transformer以融合多模态数据。Triformer包括：I) 图像Transformer，用于从医学扫描中提取多视角图像特征；II) 临床Transformer，用于嵌入并关联多模态临床数据；III) 模态融合Transformer，通过融合图像与临床Transformer的输出结果生成精确预测。在阿尔茨海默病神经影像学倡议（ADNI1）和ADNI2数据集上的评估表明，Triformer的性能超越了先前最先进的单模态及多模态方法。