Glaucoma is a chronic neurodegenerative condition that can lead to blindness. Early detection and curing are very important in stopping the disease from getting worse for glaucoma patients. The 2D fundus images and optical coherence tomography(OCT) are useful for ophthalmologists in diagnosing glaucoma. There are many methods based on the fundus images or 3D OCT volumes; however, the mining for multi-modality, including both fundus images and data, is less studied. In this work, we propose an end-to-end local and global multi-modal fusion framework for glaucoma grading, named ELF for short. ELF can fully utilize the complementary information between fundus and OCT. In addition, unlike previous methods that concatenate the multi-modal features together, which lack exploring the mutual information between different modalities, ELF can take advantage of local-wise and global-wise mutual information. The extensive experiment conducted on the multi-modal glaucoma grading GAMMA dataset can prove the effiectness of ELF when compared with other state-of-the-art methods.

翻译：青光眼是一种慢性神经退行性疾病，可致失明。对青光眼患者而言，早期检测与治疗对阻止病情恶化至关重要。二维眼底图像和光学相干断层扫描（OCT）有助于眼科医生诊断青光眼。目前已有许多基于眼底图像或三维OCT体数据的方法，然而对涵盖眼底图像与OCT数据的多模态信息挖掘研究仍较少。本文提出一种名为ELF的端到端局部与全局多模态融合框架用于青光眼分级。ELF能充分利用眼底图像与OCT之间的互补信息。不同于以往将多模态特征简单拼接、缺乏探索模态间互信息的做法，ELF可同时利用局部与全局互信息。在用于多模态青光眼分级的GAMMA数据集上进行的广泛实验表明，与其他最先进方法相比，ELF的有效性得到了验证。